{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multi-label prediction with Planet Amazon dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from fastai.vision.all import *\n",
    "from nbdev.showdoc import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Getting the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The planet dataset isn't available on the [fastai dataset page](https://course.fast.ai/datasets) due to copyright restrictions. You can download it from Kaggle however. Let's see how to do this by using the [Kaggle API](https://github.com/Kaggle/kaggle-api) as it's going to be pretty useful to you if you want to join a competition or use other Kaggle datasets later on.\n",
    "\n",
    "First, install the Kaggle API by uncommenting the following line and executing it, or by executing it in your terminal (depending on your platform you may need to modify this slightly to either add `source activate fastai` or similar, or prefix `pip` with a path. Have a look at how `conda install` is called for your platform in the appropriate *Returning to work* section of https://course.fast.ai/. (Depending on your environment, you may also need to append \"--user\" to the command.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ! {sys.executable} -m pip install kaggle --upgrade"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then you need to upload your credentials from Kaggle on your instance. Login to kaggle and click on your profile picture on the top left corner, then 'My account'. Scroll down until you find a button named 'Create New API Token' and click on it. This will trigger the download of a file named 'kaggle.json'.\n",
    "\n",
    "Upload this file to the directory this notebook is running in, by clicking \"Upload\" on your main Jupyter page, then uncomment and execute the next two commands (or run them in a terminal). For Windows, uncomment the last two commands."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ! mkdir -p ~/.kaggle/\n",
    "# ! mv kaggle.json ~/.kaggle/\n",
    "\n",
    "# For Windows, uncomment these two commands\n",
    "# ! mkdir %userprofile%\\.kaggle\n",
    "# ! move kaggle.json %userprofile%\\.kaggle"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You're all set to download the data from [planet competition](https://www.kaggle.com/c/planet-understanding-the-amazon-from-space). You **first need to go to its main page and accept its rules**, and run the two cells below (uncomment the shell commands to download and unzip the data). If you get a `403 forbidden` error it means you haven't accepted the competition rules yet (you have to go to the competition page, click on *Rules* tab, and then scroll to the bottom to find the *accept* button)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Path('/home/sgugger/.fastai/data/planet')"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path = Config().data/'planet'\n",
    "path.mkdir(parents=True, exist_ok=True)\n",
    "path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#! kaggle competitions download -c planet-understanding-the-amazon-from-space -f train-jpg.tar.7z -p {path}  \n",
    "#! kaggle competitions download -c planet-understanding-the-amazon-from-space -f train_v2.csv -p {path}  \n",
    "#! unzip -q -n {path}/train_v2.csv.zip -d {path}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To extract the content of this file, we'll need 7zip, so uncomment the following line if you need to install it (or run `sudo apt install p7zip-full` in your terminal)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ! conda install --yes --prefix {sys.prefix} -c haasad eidl7zip"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And now we can unpack the data (uncomment to run - this might take a few minutes to complete)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#! 7za -bd -y -so x {path}/train-jpg.tar.7z | tar xf - -C {path.as_posix()}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multiclassification"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Contrary to the pets dataset studied in last lesson, here each picture can have multiple labels. If we take a look at the csv file containing the labels (in 'train_v2.csv' here) we see that each 'image_name' is associated to several tags separated by spaces."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(path/'train_v2.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To put this in a `DataLoaders` while using the [data block API](https://docs.fast.ai/data_block.html), to do this we need to indicate:\n",
    "- the types of our inputs/targets (here image and multi-label categories) through a thing called blocks\n",
    "- how to get our xs and ys from the dataframe through a `ColReader`\n",
    "- how to split out data between training and validation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we have satellite images, it makes sense to use all kinds of flip, we limit the amount of lighting/zoom and remove the warping."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tfms = aug_transforms(flip_vert=True, max_lighting=0.1, max_zoom=1.05, max_warp=0., size=128)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "planet = DataBlock(blocks=(ImageBlock, MultiCategoryBlock),\n",
    "                   get_x=ColReader(0, pref=str(path/\"train-jpg\")+\"/\", suff='.jpg'),\n",
    "                   get_y=ColReader(1, label_delim=' '),\n",
    "                   splitter=RandomSplitter(seed=42),\n",
    "                   batch_tfms=tfms+[Normalize.from_stats(*imagenet_stats)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we have satellite images, it makes sense to use all kinds of flip, we limit the amount of lighting/zoom and remove the warping."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dls = planet.dataloaders(df, bs=64, path=path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`show_batch` still works, and show us the different labels separated by `;`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dls.show_batch(max_n=9, figsize=(12,9))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To create a `Learner` we use the same function as in lesson 1. Our base architecture is resnet50 again, but the metrics are a little bit differeent: we use `accuracy_thresh` instead of `accuracy`. In lesson 1, we determined the predicition for a given class by picking the final activation that was the biggest, but here, each activation can be 0. or 1. `accuracy_thresh` selects the ones that are above a certain threshold (0.5 by default) and compares them to the ground truth.\n",
    "\n",
    "As for Fbeta, it's the metric that was used by Kaggle on this competition. See [here](https://en.wikipedia.org/wiki/F1_score) for more details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "arch = resnet50"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "acc_02 = partial(accuracy_multi, thresh=0.2)\n",
    "f_score = FBetaMulti(2, thresh=0.2, average='samples')\n",
    "learn = cnn_learner(dls, arch, metrics=[acc_02, f_score])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use the LR Finder to pick a good learning rate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxU5dn/8c+VnYRskBBCEiAgW0TZIq5VrIpoLaDVVtRWffzJU5faVn1au7y62Men1tpq3WqtVWurUNTaosWiVXBHCULYlxCWBAIEsu/b9ftjBh1hskBy5mRmrvfrNS9n7rN9jWOu3Oc+5z6iqhhjjDFHinA7gDHGmP7JCoQxxhi/rEAYY4zxywqEMcYYv6xAGGOM8csKhDHGGL+inNqxiDwFXAIcUNWJfpaPB54GpgI/UtX7fZbNAn4HRAJPquq93R0vLS1NR44c2UfpjTEmPKxateqgqqb7W+ZYgQCeAR4Bnu1keQVwGzDXt1FEIoFHgQuAUmCliCxW1Y1dHWzkyJEUFBT0NrMxxoQVEdnV2TLHTjGp6jt4ikBnyw+o6kqg9YhF04EiVS1W1RZgITDHqZzGGGP8649jEFlAic/nUm/bUURkvogUiEhBeXl5QMIZY0y46I8FQvy0+Z0PRFWfUNV8Vc1PT/d7Cs0YY8xx6o8FohTI8fmcDex1KYsxxoSt/lggVgJjRCRXRGKAK4HFLmcyxpiw4+RlrguAGUCaiJQCPwWiAVT1cREZChQASUCHiHwHyFPVGhG5FViK5zLXp1R1g1M5jTHG+OdYgVDVed0s34fn9JG/ZUuAJU7kClZVDS3sOtTAicOSiIrsjx0/Y0yocfI+CONHbVMrr63bR0VDC7VNrdQ2tXGoroUDtU0cqG1mQHQkP7h4AueM/WzQ/f2ig3x74RoO1jWTEh/NjLHpnDchg/yRqQxNikPks3H9xpZ2Nu+rYU1JFat3V7GmpIrmtnZy0xLITRvI6PQEvjh+CKPSB7rxr2+MCSISKg8Mys/P1/5+o1xjSztf/9NHFOyqBCBCIDEumsEJMaQnxjIkKY4Ne6opPljP3MnD+OGXJvDXFbt5+K1tjE4fyPyzR/FRcQXLthygor4FgLSBMUzMSiY+JpLNZbXsOFTP4f+kmclxTBmewoDoKHYeqmfHwfpPtxs/NJFLTs5kzuQscgbFu/LzMMa4T0RWqWq+32VWIAKjrb2Db/51FW9uPsCDX5vM+RMyiI+J/Nxf/wBNre08tnw7v19eRIdCe4dy+bRs7p5zIvExng5fe4eytrSKwpIq1u2pYf2eapra2hk/NJHxQ5OYkJnIpJwUMpMHHJVjb1Ujr63fx5J1ZazaVYkIzBibztdPH8E5Y4fQ2t5BaWUDuysaqGtup72jg7Z2z3ckITaKhNgoBsZGMjYjkcS4aOd/cMYYR1mBcJmq8v2X1rKooJRfzJ3I108b0e022/bX8sB/tnL+hAwum+p3qKbX9lQ18reVJSz4eDfltc0kxkZR19JGT74SkRHClJwUvjAmnTNPGMzErGTioiMdyWmMcY4ViF6obmylvLaZyoYWKutbGD80ieGDe35KZm9VI48uK+K5j3Zz23ljuP2CsX2esbda2ztYumEf7xcdJCMpjhGD4xk+KIHkAdFERQiREYIqNLS2Ud/cRnVjK6t2VfLetoOs3VONKkRHCnnDkpmSk8LIwfEMSYojIymWlPgYYiIjiImKIDYqgpT4GLf/dY0xPqxAHKM9VY28tq6M19bvY5V3vOCwxNgoXrr5DMZmJHa6fUeH8q91ZSwqKOG9ooOownVnjOSnX8476pRSsKusb+HjnRWs3l3F6t2VrC2tprG1vdP1s1IGcNqowZw+ejDjhyYSHxNJfEwUA+OiGBjb+TUT7R3Khr3VfFRcQWSEMCEzibzMJJLj7TSXMb1hBeIY3L90C48sKwIgLzOJC08cysi0eFLjY4iOjOC2hauJjYrgH7ecSdrAWL/7uPe1zTz+9nayUgbwlWnZXD41+5h6HcGso0OpbGjhQG0z+2uaqG5spbVdaW3voLapldW7q1hRfIjKhiPnaPQUj0k5yZycnULygGgO1jZzqL6FkooGPt5ZQW1T21HbZCbHkZMaT1bqALJSBpAQG0VkBESIkBQXTf7IVHLTEkKuMBvTV6xA9ND6PdXMfuQ9LpqYyf9cOI6RaQlHrVNYUsXXnviQCZlJLLjxtKPOu7+5aT83/LmAedNzuGfuSURE2C+mI3V0KFsP1LLrUAONLe00tLRT1djChr01rC2toqSi8dN1E+OiyEiK45SRqZ/2PAA2ldWycW8N2/bXUlrZyJ6qRsqqG+nw83XOSIrltFGDmZKTwknZyeRlJjMgxsZLjAErED3S0aF85fEP2H2ogbfumNHlqYvX1pVx03OfcMnJmTzwtclEe29cK6lo4JKH3yM7dQAv3XSGDdoep8r6Fhpb2xk8MIbYqJ7/DNvaO2htVzpUaVelvLaZj4or+LD4ECuKD1Fe2wx4Li8+YchAJmQmMX5oEuOHJpKblsCwlAHERNlNiCa8dFUg7EY5rxdWlbB6dxX3XzGp2/PaF52UyfdmjeO+f2/hk12VXH9mLpdNzeKW5z+hQ5XHrp5qxaEXUhNiSD2O7aIiI/CtJ0lx0YxOH8hVpw5HVdlX08S60mrW76lm/d4aCnZW8s81n80DGSGQmTyAzOQ4UuJjSImPJnlANBECHQodqowYFM+cyVmkJthguwl91oPAM43Fufcv54QhA1n036f36Hy1qvLW5gM88U4xH+3wDJy2dyiPXzONWROHHlcOE3jVDa1s2V/LrkP1lFR47v/YV9NEVUMr1Y2eF3jGNADqmtuIiYzgwolDuWxqFjmp8STFRZE0INr+KDBByXoQ3bhv6RZqmtq4e87EHg9migjnTcjgvAkZrC2t4pkPdjI2I9GKQ5BJjo9meu4gpucO6tH6G/fWsKighJdX7+GVws/PQj8pO5kfXjyBU0cNdiKqMQEX9j2I7eV1nP/bt7n+jFx+8uU8B5KZUNTU2s7HOyqobGihpqmNiroWFq7cTVl1ExeemMFdF00g189FDsb0NzZI3Y03N+1neu4gmzrC9EpjSztPvlvM79/eTktbB3OnZHHTjNGMtokRTT9mBcKYADpQ28Rjy7azcOVumts6uHhiJrfPHGuFwvRLViCMccHBumaeem8Hz364iw5V7rl0IpdOcWZeLWOOV1cFwi76NsYhaQNj+d6s8fzn9nOYOCyZ7/6tkLteWktTF1ORGNOfWIEwxmFDk+N4/sZTuXnGaBauLGH2I+/x+oZ9hErv3YQuxwqEiDwlIgdEZH0ny0VEHhKRIhFZKyJTfZa1i8ga72uxUxmNCZSoyAi+N2s8z1x/Ck2tHcz/yyou+t27/GttGR3+5gcxph9wsgfxDDCri+UXAWO8r/nA732WNarqZO9rtnMRjQmsGeOG8NYd5/Dbr06ipb2DW57/hEt//wHr91S7Hc2YozhWIFT1HaCii1XmAM+qxwogRUQyncpjTH8RFRnBZVOzeeO7nkKxp7KR2Y+8x89f2UBt09Gz3BrjFjfHILKAEp/Ppd42gDgRKRCRFSIyt7MdiMh873oF5eXlTmY1ps9FRgiXTc3mzTvO4epTR/DMBzu54Lfv8J+N+92OZgzgboHwN6fF4ZOxw72XXV0FPCgio/3tQFWfUNV8Vc1PT093KqcxjkoeEM0v5k7k7zedQUp8NP/v2QK+tWA1B+ua3Y5mwpybBaIUyPH5nA3sBVDVw/8sBpYDUwIdzphAmzI8lcW3nsXtF4zl3+vLuOC3b/PSqlK72sm4xs0CsRj4hvdqptOAalUtE5FUEYkFEJE04Exgo4s5jQmYmKgIbjtvDEtu+wK5aQnc8UIhV/3xI4rL69yOZsKQk5e5LgA+BMaJSKmI3CAi3xSRb3pXWQIUA0XAH4Gbve0TgAIRKQSWAfeqqhUIE1bGZCTy4jfP4J5LJ7J+bzWzHnyXh97cRmt7h9vRTBixqTaM6ecO1DZx9ysbeXVtGZOyk/nNVydzwhCb18n0DZtqw5ggNiQxjkeumspjV09lV0UDX3roXZ55f4fdYGccZwXCmCBx8UmZvP6dszlj9GB+9spGbvjzSg7ZlU7GQVYgjAkiQ5LieOq6U/jFnBN5f/shLvrdu3yw/aDbsUyIsgJhTJAREb5++kj+cfOZDIyL4uonP+Le1zZT39zmdjQTYqxAGBOk8oYl8eq3zuKKadk8/vZ2Zty/nEUrS2i3sQnTR6xAGBPE4mOiuO/ySfz95jPISR3A915ayyUPv2eT/5k+YQXCmBAwdXgqL910Bg/Pm0JFfTOXPvY+j7+93XoTplesQBgTIkSEL08axtLvnM35EzK497XNXP3kCvZWNbodzQQpKxDGhJiU+Bgeu3oq911+MmtLq5n5wDs88/4O602YY2YFwpgQJCJ8NT+Hf3/7bKYMT+Fnr2zkssfeZ8NeG5swPWcFwpgQNnxwPM/+13QemjeFPVWNzH7kfRYX7nU7lgkSViCMCXEiwuxJw3jz9hlMG5HKdxau5p9r9rgdywQBKxDGhInk+Gieuf4UpucO4rt/W8M/VluRMF2zAmFMGImPieKp607h1NzB3L5oDX//pNTtSKYfswJhTJg5XCROHz2YO14o5PmPdrsdyfRTViCMCUMDYiL507WnMGNsOj98eR1/em+H25FMP2QFwpgwFRcdyR++ns9FE4fyi1c38shb29yOZPoZKxDGhLGYqAgenjeFy6Zkcf/rW+3qJvM5ViCMCXNRkRHcd/nJ5I9I5Yd/X8fOg/VuRzL9hGMFQkSeEpEDIrK+k+UiIg+JSJGIrBWRqT7LrhWRbd7XtU5lNMZ4REVG8Lt5U4iKjODWBZ/Q3NbudiTTDzjZg3gGmNXF8ouAMd7XfOD3ACIyCPgpcCowHfipiKQ6mNMYA2SlDOD+Kyaxfk8N97622e04ph9wrECo6jtARRerzAGeVY8VQIqIZAIXAm+oaoWqVgJv0HWhMcb0kQvyMrjujJE8/f5OXt+wz+04xmVujkFkASU+n0u9bZ21H0VE5otIgYgUlJeXOxbUmHDyg4vHMzEriTtfKKSkosHtOMZFbhYI8dOmXbQf3aj6hKrmq2p+enp6n4YzJlzFRkXy2FXTUOCW5208Ipy5WSBKgRyfz9nA3i7ajTEBMnxwPL++fBJrS6v5v39tcjuOcYmbBWIx8A3v1UynAdWqWgYsBWaKSKp3cHqmt80YE0CzJg7l/52Vy58/3MWra+1vtHAU5dSORWQBMANIE5FSPFcmRQOo6uPAEuBioAhoAK73LqsQkV8AK727ultVuxrsNsY45PsXjeeT3ZV8/8W1nDBkIOOHJrkdyQSQqIbGYwjz8/O1oKDA7RjGhJx91U3MefQ9oiIiePmWMxiSGOd2JNOHRGSVqub7W2Z3UhtjujQ0OY4/XXsKFfUtzH92FU2tNmgdLqxAGGO6NTErmQevnExhaRV3vFBIR0donHkwXbMCYYzpkQtPHMpds8bzr7Vl/P7t7W7HMQFgBcIY02Pzzx7FJSdn8sAbW1lXWu12HOMwKxDGmB4TEe6ZexJpA2P5zt9W23hEiLMCYYw5Jsnx0dx/xSS2l9fbpH4hzgqEMeaYnTUmjevPHMkzH+zkna02D1qosgJhjDku3581njFDBnLHC4Xsq25yO45xgBUIY8xxiYuO5JGrptLQ3Mb8vxTYeEQIsgJhjDlu44Ym8sDXJrNuTzXfe3EtoTIzg/GwAmGM6ZWZJw7lzpnjWFy41+6PCDFWIIwxvXbzjNF8edIwfr10C8u2HHA7jukjViCMMb0mItz3lZMZl5HInYsKOVBrg9ahwAqEMaZPDIiJ5KF5U6hrbuPOF9bafE0Bcuvzn3DtUx87sm8rEMaYPjM2I5EfX5LHO1vLefqDnW7HCQv7qptobe9wZN9WIIwxfeqaU4dz/oQMfvXaZjbstfmanFbV2EpKfLQj+7YCYYzpUyLCfZefTEp8NN9euMbuj3BYVUMryQOsQBhjgsSghBh+fcUkig7U8fBb29yOE7JUlZrGVpIHxDiyf0cLhIjMEpEtIlIkInf5WT5CRN4UkbUislxEsn2WtYvIGu9rsZM5jTF975yx6VwxLZvH3y5m/R471eSExtZ2Wto7gu8Uk4hEAo8CFwF5wDwRyTtitfuBZ1X1ZOBu4Jc+yxpVdbL3NdupnMYY5/z4S3kMTojhzhcKaWlzZiA1nFU3tgIE5Smm6UCRqharaguwEJhzxDp5wJve98v8LDfGBLHk+GjuufQkNu+r5XG7y7rPVTV4CkRKEBaILKDE53Opt81XIfAV7/tLgUQRGez9HCciBSKyQkTm+juAiMz3rlNQXm5TDhvTH12Ql8HsScN4+K1tbN1f63ackHK4QARjD0L8tB1558ydwDkisho4B9gDtHmXDVfVfOAq4EERGX3UzlSfUNV8Vc1PT0/vw+jGmL70s9knkhAbxU/+ud4m9OtDn55iCrYxCDw9hhyfz9nAXt8VVHWvql6mqlOAH3nbqg8v8/6zGFgOTHEwqzHGQYMSYrhz5jhWFFfw6toyt+OEjOrGFgBS4oPvKqaVwBgRyRWRGOBK4HNXI4lImogczvAD4Clve6qIxB5eBzgT2OhgVmOMw+ZNH87ErCTu+dcm6pvbut/AdCtoTzGpahtwK7AU2AQsUtUNInK3iBy+KmkGsEVEtgIZwD3e9glAgYgU4hm8vldVrUAYE8QiI4Sfz57IvpomHrJ7I/pEdWMrURFCQkykI/uPcmSvXqq6BFhyRNtPfN6/CLzoZ7sPgJOczGaMCbxpI1K5fFo2T723gyum5XDCkIFuRwpqh6fZEPE35Nt7die1MSagvj9rPHHRkfziVTsp0FvVja0kOXR6CaxAGGMCLD0xllvPPYG3t5azoviQ23GCWnVDq2P3QIAVCGOMC649YyQZSbHcv3SLXfbaC1WNLY4NUIMVCGOMC+KiI/nWF8dQsKuS5VvsJtfjVd3Y6tglrmAFwhjjkq/m5zB8UDy/XrrFnj53nJyc6husQBhjXBITFcF3LxjDxrIalqy3m+eOVXuHUtvUZgXCGBOaZk/KYmzGQH77+lbaHHpsZqiq8U6z4dRU32AFwhjjosgI4Y6Z4yg+WM8/1uztfgPzqSqHp/oGKxDGGJfNzMsgLzOJR97aZr2IY1DdX3oQIjLaZ26kGSJym4ikOJbKGBM2RIRvnz+GnYca+Kf1InqsqsEzUZ9TjxuFnvcgXgLaReQE4E9ALvC8Y6mMMWFlZl4GEzKTeGRZkfUiesjpp8lBzwtEh3fyvUuBB1X1u0CmY6mMMWFFRPj2eSew42A9r6y1XkRP9JtTTECriMwDrgVe9bY5l8oYE3Zm5g1l/NBEHn6ziHa7L6JbTk/1DT0vENcDpwP3qOoOEckF/upYKmNM2ImIEL593hiKD9bzSqH1IrpT3dhKQkwk0ZHOXWvUoz2r6kZVvU1VF4hIKpCoqvc6lsoYE5YuPHEo4zISeWx5kc3R1I2qBmen2YCeX8W0XESSRGQQUAg8LSK/dTSZMSbsREQIN549iq3763hn20G34/RrTk/1DT0/xZSsqjXAZcDTqjoNON+5WMaYcDV70jCGJMby5LvFbkfp16obWxyd6ht6XiCiRCQT+CqfDVIbY0yfi4mK4NozRvLutoNsKqtxO06/5fREfdDzAnE3nmdLb1fVlSIyCuj2obIiMktEtohIkYjc5Wf5CBF5U0TWek9jZfssu1ZEtnlf1/b0X8gYE/yuPnU4A6IjefLdHW5H6beqvY8bdVJPB6lfUNWTVfUm7+diVf1KV9uISCTwKHARkAfME5G8I1a7H3hWVU/GU4R+6d12EPBT4FRgOvBT7+C4MSYMpMTH8NX8bBYX7uFATZPbcfodVaWqsZXk/lAgRCRbRF4WkQMisl9EXvL9a78T04EibzFpARYCc45YJw940/t+mc/yC4E3VLVCVSuBN4BZPclqjAkN15+ZS1uH8ucPd7odpd9pau2gpa2j35xiehpYDAwDsoBXvG1dyQJKfD6Xett8FQKHeyKXAokiMriH2yIi80WkQEQKysvtqVTGhJKRaQnMzMvgryt209DS5nacfuXTu6gdnIcJel4g0lX1aVVt876eAdK72Ub8tB15YfOdwDkisho4B9gDtPVwW1T1CVXNV9X89PTu4hhjgs0NZ42iurHVbpw7QlXj4Yn6+kcP4qCIXCMikd7XNcChbrYpBXJ8PmcDn/uvrKp7VfUyVZ0C/MjbVt2TbY0xoe+UkamMy0jkryt2ux2lX6lucH4eJuh5gfgvPJe47gPKgMvxTL/RlZXAGBHJFZEY4Eo8p6k+JSJpInI4ww+Ap7zvlwIzRSTVOzg909tmjAkjIsLVpw1n3Z5qCkuq3I7TbwTiYUHQ86uYdqvqbFVNV9UhqjoXz01zXW3TBtyK5xf7JmCRqm4QkbtFZLZ3tRnAFhHZCmQA93i3rQB+gafIrATu9rYZY8LMpVOyiI+J5LmPdrkdpd+oDsBEfQBRvdj2duDBrlZQ1SXAkiPafuLz/kXgxU62fYrPehTGmDCVGBfNnMlZvLy6lB9dnOf4pZ3BIBBTfUPvHjnqbyDZGGP63NWnDqeptYOXPil1O0q/UNXYQmSEMDC2N3/jd683BcKmWjTGBMTErGQm56Tw3Ee7bJZXPD2I5AHRiDj7d3qXBUJEakWkxs+rFs89EcYYExDXnDaC7eX1rCi24ciqhlbHJ+qDbgqEqiaqapKfV6KqOtu3McYYH5ecnEnygGie/9gueQ3EVN/Qu1NMxhgTMHHRkcydPIylG/Z9ehVPuArERH1gBcIYE0SuyM+hpa2DxWvD+77ZfnGKyRhj+pMThyUxfmgiLxaUdL9yCDs8SO00KxDGmKAhIlyRn0NhaTVb99e6HccV7R1KTVMryQ4/jxqsQBhjgszcycOIihBeCNNeRG1TK6rO30UNViCMMUFm8MBYzpswhJdX76G1vcPtOAH32VTfViCMMeYoV0zL4WBdC8s2H3A7SsBVBmgmV7ACYYwJQjPGpZM2MJYXVoXf1BsV9c0ADEqwMQhjjDlKVGQEl03NYtnmAxysa3Y7TkAdqvM8LGhwQqzjx7ICYYwJSldMy6atQ/nH6j1uRwmoinpPgRg00HoQxhjj15iMRKYMT+FvK0vCagK/ivoWYqIiSIiJdPxYViCMMUHrimk5bDtQR2FptdtRAuZQfQuDE2Icn8kVrEAYY4LYJZMyiYuOCKt7IirqW0gNwE1yYAXCGBPEkuKiuXhiJovX7KWxpd3tOAFRUd/C4ACMP4DDBUJEZonIFhEpEpG7/CwfLiLLRGS1iKwVkYu97SNFpFFE1nhfjzuZ0xgTvK7Iz6G2uY2lG/a5HSUgKupbAnKJKzhYIEQkEngUuAjIA+aJSN4Rq/0YWKSqU4Argcd8lm1X1cne1zedymmMCW6n5g5i+KB4FoXJaaaQKBDAdKBIVYtVtQVYCMw5Yh0Fkrzvk4HwnsPXGHPMIiKEy6dl88H2Q5RUNLgdx1HNbe3UNbcxOAQKRBbgW9JLvW2+fgZcIyKlwBLgWz7Lcr2nnt4WkS/4O4CIzBeRAhEpKC8v78Poxphg8pVp2YgQ8oPVn94DEYCb5MDZAuHvGqwjL1aeBzyjqtnAxcBfRCQCKAOGe0893Q48LyJJR2yLqj6hqvmqmp+ent7H8Y0xwSIrZQDnjE1n4cqSkJ7A7/Bd1IMSnJ+HCZwtEKVAjs/nbI4+hXQDsAhAVT8E4oA0VW1W1UPe9lXAdmCsg1mNMUHumlNHcKC2mTc37Xc7imNCqQexEhgjIrkiEoNnEHrxEevsBs4DEJEJeApEuYikewe5EZFRwBig2MGsxpggd+74IQxLjuO5j3a7HcUxlQ2HC0SQj0GoahtwK7AU2ITnaqUNInK3iMz2rnYHcKOIFAILgOvUc8/82cBab/uLwDdVtcKprMaY4BcZIVw5fTjvbjvIjoP1bsdxxGcT9QWmQEQ5uXNVXYJn8Nm37Sc+7zcCZ/rZ7iXgJSezGWNCz5Wn5PC7N7ex4OPd/PDiCW7H6XMV9S1ERkhAniYHdie1MSaEDEmKY2ZeBi8UlNDUGnp3Vh+qbyE1PpqICOfnYQIrEMaYEHPNaSOobGjltfVlbkfpcxX1zQEbfwArEMaYEHP6qMHkpiXw1xWhN1gdyIn6wAqEMSbEREQIV00fzqpdlWwqq3E7Tp86FMCJ+sAKhDEmBF0+LZuYqAieD7FLXisDOA8TWIEwxoSg1IQYLjkpk5dX76G+uc3tOH2ivUOpamwN2E1yYAXCGBOirj5tBHXNbfxzTWjMAVrZ0IJq4O6BACsQxpgQNXV4CuOHJvLXFbtC4pnVn02zYQXCGGN6RUS4+rQRbCyrYU1Jldtxei3Qd1GDFQhjTAi7dEoWCTGRITE/0+EeRKoVCGOM6b2BsVHMmZLFK4V7qfJOdBesKhqsB2GMMX3qmlNH0NzWwd9WBvfDhCrqrAdhjDF9Km9YEmedkMYf391BY0vwzs9UUd9MUlwU0ZGB+7VtBcIYE/JuO28MB+uaWfBx8I5FeO6iDtw9EGAFwhgTBqbnDuLU3EH84Z3tQTvLa0WA76IGKxDGmDDx7fPGsL+mmRcKgnMswgqEMcY45PTRg8kfkcpjy7fT3BZ8vYhD9S0MCuBMrmAFwhgTJkSE284bQ1l1Ey+t2uN2nGOiqp6J+gI4kys4XCBEZJaIbBGRIhG5y8/y4SKyTERWi8haEbnYZ9kPvNttEZELncxpjAkPXxiTxuScFB5bXkRbe4fbcXqspqmNtg4N6D0Q4GCBEJFI4FHgIiAPmCcieUes9mNgkapOAa4EHvNum+f9fCIwC3jMuz9jjDluIsIt555AaWUj/1oXPE+cc2MeJnC2BzEdKFLVYlVtARYCc45YR4Ek7/tk4PC0i3OAhararKo7gCLv/owxplfOGz+EE4YM5PG3i4NmEr+K+mYgtApEFuB7uUCpt83Xz4BrRKQUWAJ86xi2RUTmi0iBiBSUl5f3VW5jTAiLiBDmnz2KTWU1vL01OH5vfDZRX+jcByF+2o4s1/OAZ1Q1G7gY+IuIRPRwW1T1CVsdk7cAAA7gSURBVFXNV9X89PT0Xgc2xoSHuZOzGJoUx+Nvb3c7So98eoophAapS4Ecn8/ZfHYK6bAbgEUAqvohEAek9XBbY4w5LjFREdxwVi4riiuCYirwQ4cLRAhd5roSGCMiuSISg2fQefER6+wGzgMQkQl4CkS5d70rRSRWRHKBMcDHDmY1xoSZeacOJykuij8EQS+isr6FAdGRDIgJ7LU6jhUIVW0DbgWWApvwXK20QUTuFpHZ3tXuAG4UkUJgAXCdemzA07PYCPwbuEVVg+/OFmNMvzUwNoqvnz6Cf2/YR3F5ndtxunSgtpnBAT69BA7fB6GqS1R1rKqOVtV7vG0/UdXF3vcbVfVMVZ2kqpNV9XWfbe/xbjdOVV9zMqcxJjxdd0YuMZER/H55/+5FbN1fy9iMxIAf1+6kNsaErfTEWOZNH87Lq/dQUtHgdhy/mtvaKTpQx4RMKxDGGBNQ/33OKCJE+u0VTdv219HWoeRlJgf82FYgjDFhLTN5AJfnZ/NCQSn7qpvcjnOUjWU1ANaDMMYYN9x0zmg6VPnDO/2vF7Fxbw3xMZGMGJwQ8GNbgTDGhL2cQfFcOiWL5z/aTXlts9txPmdjWQ3jhyYSGeHv/mFnWYEwxhjg5nNPoLW9gyffLXY7yqdUlU1lNUzITOp+ZQdYgTDGGCA3LYE5k7N45oOd/eaKptLKRmqb2sgbZgXCGGNc9b1Z44iMEO5+daPbUYDPBqjzrAdhjDHuykwewLe+OIY3Nu5n2ZYDbsdhU1kNIjBuaOCvYAIrEMYY8zk3nJXLqLQEfr54g+vPrt64t4bctATiY6JcOb4VCGOM8RETFcHPZp/IzkMNPPnuDlezbHRxgBqsQBhjzFHOHpvOrBOH8vBb29hb1ehKhurGVkorG10bfwArEMYY49ePL5lAe4fy6LIiV46/+fAAtUtXMIEVCGOM8Ss7NZ4r8nNYVFDiSi/C7SuYwAqEMcZ06uYZowFcmQ58U1kNgxNiGJIY2OdQ+7ICYYwxnchOjefyadn8bWUJZdWB7UVsLKshb1gSIoGfYuMwKxDGGNOFm2ecQIcqjwewF9Ha3sHWfXWuXsEEViCMMaZLOYM8vYgFH5cEbDrw94oO0tLewdThqQE5XmesQBhjTDduOdfTi3j4rW0BOd6ilSUMSojhi+OHBOR4nXG0QIjILBHZIiJFInKXn+UPiMga72uriFT5LGv3WbbYyZzGGNOVnEHxXHPaCJ77aDdvbtrv6LEO1TXzn037mTs5i5god/+Gd+zoIhIJPApcBOQB80Qkz3cdVf2uqk5W1cnAw8DffRY3Hl6mqrOdymmMMT1x10XjyctM4vZFhZRWOjfb68ur99DarnztlBzHjtFTTpan6UCRqharaguwEJjTxfrzgAUO5jHGmOMWFx3JY1dPpb1DufX51bS0dfT5MVSVRQUlTMpJcW2CPl9OFogsoMTnc6m37SgiMgLIBd7yaY4TkQIRWSEiczvZbr53nYLy8vK+ym2MMX6NTEvgV185mTUlVfzq35v7fP9rSqrYur+Or+W733sAZwuEv4t3tZN1rwReVFXfqROHq2o+cBXwoIiMPmpnqk+oar6q5qenp/c+sTHGdONLJ2fyjdNH8Kf3dvDO1r79w3RRQQkDoiP58qTMPt3v8XKyQJQCvmUwG9jbybpXcsTpJVXd6/1nMbAcmNL3EY0x5tj98OIJjEpP4Icvr6O+ua1P9tnQ0sYrhWVcfFImiXHRfbLP3nKyQKwExohIrojE4CkCR12NJCLjgFTgQ5+2VBGJ9b5PA84E+scjnowxYS8uOpJ7LzuZ0spGfvP61j7Z56try6hrbusXg9OHOVYgVLUNuBVYCmwCFqnqBhG5W0R8r0qaByxUVd/TTxOAAhEpBJYB96qqFQhjTL8xPXcQ15w2nKc/2MHq3ZW92ld9cxsPvrGVCZlJnDLS3ZvjfMnnfy8Hr/z8fC0oKHA7hjEmjNQ2tTLzgXdIiovmlW+dddz3Lfzvqxt58r0dvHTT6UwbMaiPU3ZNRFZ5x3uPYndSG2PMcUqMi+Z/505ky/5afnqcjyjdsLeapz/YybzpwwNeHLpjBcIYY3rhvAkZ3PiFXBZ8vJsvP/we60qre7xte4fyw5fXkxofzV2zxjuY8vhYgTDGmF760ZfyePr6U6hubGXuY+/z29e39OhGuuc/3k1hSRU//lIeyfH948olX1YgjDGmD5w7bgivf+cc5kwexkNvFTHn0ffZsLfz3kRhSRW/em0zZ52QxpzJwwKYtOesQBhjTB9Jjo/mt1+dzB+/kc/BumbmPPI+v/vPNlrbP9+bWL27kmue/IhBCTHcd/nJrj4UqCtRbgcwxphQc0FeBvkjUvnZKxt44D9b+dvK3Vxz+gjmnTKc4oP1XPvUxwweGMOCG09jWMoAt+N2yi5zNcYYBy3fcoA/vlvM+0WHiI2KIDJCGJIYy4L5p5GZ7H5x6OoyV+tBGGOMg2aMG8KMcUPYsq+WZz7YyY6DdTz4tSkMTY5zO1q3rEAYY0wAjBuayC8vO8ntGMfEBqmNMcb4ZQXCGGOMX1YgjDHG+GUFwhhjjF9WIIwxxvhlBcIYY4xfViCMMcb4ZQXCGGOMXyEz1YaIlAO7vB+Tgeou3vtrSwMOHuNhfffT02VHtneX1V/uvs7a2fKusvrLFQxZ/WV0O2tn+fxl9W0L5+9sMGXtbHl/+c6OUNV0v2uqasi9gCe6et9JW0FvjtPTZUe2d5fVX+6+ztrZ8q6y9uDn2S+zdpLR1azH8t/eybzB9J0Npqw9/R70l++s7ytUTzG90s37zpb35jg9XXZke3dZfd87lbWz5V1l9f0cTFl93/eXrEe22Xe2+22DKWtny/vrd/ZTIXOKqbdEpEA7mdGwv7GszgimrBBceS2rM5zOGqo9iOPxhNsBjoFldUYwZYXgymtZneFoVutBGGOM8ct6EMYYY/yyAmGMMcYvKxDGGGP8sgJhjDHGLysQ3RCRL4jI4yLypIh84Hae7ohIhIjcIyIPi8i1bufpiojMEJF3vT/fGW7n6Y6IJIjIKhG5xO0sXRGRCd6f6YsicpPbebojInNF5I8i8k8Rmel2nq6IyCgR+ZOIvOh2Fn+839E/e3+eV/d2fyFdIETkKRE5ICLrj2ifJSJbRKRIRO7qah+q+q6qfhN4Ffhzf88LzAGygFagtJ9nVaAOiAuCrADfBxY5k/LTTH3xnd3k/c5+FXD0ev4+yvsPVb0RuA74Wj/PWqyqNziV0Z9jzH0Z8KL35zm71wc/1tu0g+kFnA1MBdb7tEUC24FRQAxQCOQBJ+EpAr6vIT7bLQKS+nte4C7gv73bvtjPs0Z4t8sAnuvnWc8HrsTzS+yS/pzVu81s4APgqv7+nfXZ7jfA1CDJ6tj/W73M/QNgsned53t77ChCmKq+IyIjj2ieDhSpajGAiCwE5qjqLwG/pw5EZDhQrao1Dsbtk7wiUgq0eD+29+esPiqBWCdyQp/9XM8FEvD8T9goIktUtaM/ZvXuZzGwWET+BTzf1zn7Mq+ICHAv8JqqftKfs7rhWHLj6YlnA2vogzNEIV0gOpEFlPh8LgVO7WabG4CnHUvUtWPN+3fgYRH5AvCOk8H8OKasInIZcCGQAjzibLSjHFNWVf0RgIhcBxx0ojh04Vh/rjPwnGqIBZY4msy/Y/3OfgtPDy1ZRE5Q1cedDHeEY/3ZDgbuAaaIyA+8hcQNneV+CHhERL5E7+ZrAsKzQIifti5vJ1fVnzqUpSeOKa+qNuApaG441qx/x1PQ3HDM3wMAVX2m76N061h/rsuB5U6F6YFjzfsQnl9sbjjWrIeAbzoXp8f85lbVeuD6vjpISA9Sd6IUyPH5nA3sdSlLTwRTXsvqjGDKCsGVN5iy+gpI7nAsECuBMSKSKyIxeAYeF7ucqSvBlNeyOiOYskJw5Q2mrL4CkztQI/FuvIAFQBmfXfJ5g7f9YmArnqsAfuR2zmDMa1kta7DlDaas/SW3zeZqjDHGr3A8xWSMMaYHrEAYY4zxywqEMcYYv6xAGGOM8csKhDHGGL+sQBhjjPHLCoQJaSJSF+DjPSkieX20r3YRWSMi60XkFRFJ6Wb9FBG5uS+ObQxg90GY0CYidao6sA/3F6WqbX21v26O9Wl2EfkzsFVV7+li/ZHAq6o6MRD5TOizHoQJOyKSLiIvichK7+tMb/t0EflARFZ7/znO236diLwgIq8Ar4vnSXjLxfPEts0i8px3ymq87fne93XiebpfoYisEJEMb/to7+eVInJ3D3s5H+KZwRMRGSgib4rIJyKyTkTmeNe5Fxjt7XX82rvu/3iPs1ZEft6HP0YTBqxAmHD0O+ABVT0F+ArwpLd9M3C2qk4BfgL8n882pwPXquoXvZ+nAN/B83yIUcCZfo6TAKxQ1Ul4pl6/0ef4v/Mev9sJ1kQkEjiPz+baaQIuVdWpwLnAb7wF6i5gu6pOVtX/Ec/jO8fgeXbAZGCaiJzd3fGMOSwcp/s25nwgz/tHP0CSiCQCycCfRWQMnimfo322eUNVK3w+f6yqpQAisgYYCbx3xHFa8DyJDGAVcIH3/enAXO/754H7O8k5wGffq4A3vO0C/J/3l30Hnp5Fhp/tZ3pfq72fB+IpGIF+TogJUlYgTDiKAE5X1UbfRhF5GFimqpd6z+cv91lcf8Q+mn3et+P//6VW/WyQr7N1utKoqpNFJBlPobkFz3MTrgbSgWmq2ioiO/E81/tIAvxSVf9wjMc1BrBTTCY8vQ7ceviDiEz2vk0G9njfX+fg8VfgObUFnmmau6Sq1cBtwJ0iEo0n5wFvcTgXGOFdtRZI9Nl0KfBfInJ4oDtLRIb00b+DCQNWIEyoixeRUp/X7Xh+2eZ7B2438tkTwu4Dfiki7+N5KLxTvgPcLiIfA5lAdXcbqOpqPA+mvxJ4Dk/+Ajy9ic3edQ4B73svi/21qr6O5xTWhyKyDniRzxcQY7pkl7kaE2AiEo/n9JGKyJXAPFWd0912xgSajUEYE3jT8DxYXoAq4L9czmOMX9aDMMYY45eNQRhjjPHLCoQxxhi/rEAYY4zxywqEMcYYv6xAGGOM8ev/A93oPj5fLBtAAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn.lr_find()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we can fit the head of our network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lr = 0.01"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy_multi</th>\n",
       "      <th>fbeta_score</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0.136436</td>\n",
       "      <td>0.116439</td>\n",
       "      <td>0.947099</td>\n",
       "      <td>0.898761</td>\n",
       "      <td>00:50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.112766</td>\n",
       "      <td>0.096898</td>\n",
       "      <td>0.951161</td>\n",
       "      <td>0.915797</td>\n",
       "      <td>00:46</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0.103073</td>\n",
       "      <td>0.091635</td>\n",
       "      <td>0.951975</td>\n",
       "      <td>0.919405</td>\n",
       "      <td>00:46</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>0.097786</td>\n",
       "      <td>0.087565</td>\n",
       "      <td>0.955528</td>\n",
       "      <td>0.922744</td>\n",
       "      <td>00:46</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>0.091133</td>\n",
       "      <td>0.087272</td>\n",
       "      <td>0.956945</td>\n",
       "      <td>0.923008</td>\n",
       "      <td>00:47</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn.fit_one_cycle(5, slice(lr))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "learn.save('stage-1-rn50')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "...And fine-tune the whole model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "learn.unfreeze()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3xV9f3H8dcni00YCSvsvRUJoCK4ULGte1StrVqts1Xbamt/7c9WWrU/be1y1Fm31j2xuMCBqIQheyQBQgAhgYQQsm7u/f7+uBdN4yW5gZyc3OT9fDzuw3vPuOfNNbnvnG3OOURERGpL8DuAiIg0TyoIERGJSgUhIiJRqSBERCQqFYSIiESlghARkaiS/A7QWNLS0tzAgQP9jiEiElcWLVpU6JxLjzauxRTEwIEDycrK8juGiEhcMbNN+xunTUwiIhKVCkJERKJSQYiISFQqCBERiUoFISIiUakgREQkKhWEiEgcW7hxF4vzijx5bxWEiEgc+/Pba5n1+ipP3lsFISISp0Ihx4otJYzLSPXk/VUQIiJxasPOvZRWVjOurwpCRERqWLFlNwDjVRAiIlLTsvzdtE1OYGh6R0/eXwUhIhKnlufvZnTvziQlevNV7mlBmNlMM1trZtlmdlOU8dPNbLGZVZvZ2bXG3WFmK81stZn93czMy6wiIvEkGHKs2Lqb8X27eLYMzwrCzBKBe4CTgdHA+WY2utZkecDFwNO15j0SmAqMB8YCk4CjvcoqIhJvNhSWUlYV9OwIJvD2fhCTgWznXC6AmT0LnAZ8dcCuc25jZFyo1rwOaAukAAYkA9s9zCoiEleW5Yd3UHt1BBN4u4kpA9hc43V+ZFi9nHMLgLnAtshjjnNude3pzOxyM8sys6yCgoJGiCwiEh+W5e+mXXIiQzzaQQ3eFkS0fQYuphnNhgKjgL6ES+U4M5v+jTdz7gHnXKZzLjM9Peod80REWqQVW3YzNqMziQne7Z71siDygX41XvcFtsY47xnAp865UudcKfAWcHgj5xMRiUvVwRArt5Yw1sP9D+BtQSwEhpnZIDNLAc4DXotx3jzgaDNLMrNkwjuov7GJSUSkNcop2Et5IOjZCXL7eFYQzrlq4MfAHMJf7s8551aa2SwzOxXAzCaZWT5wDnC/ma2MzP4CkAMsB74AvnDOve5VVhGReLIsvxiAcRneHeIK3h7FhHNuNjC71rCbazxfSHjTU+35gsAVXmYTEYlXK7bspkNKIoPTOni6HJ1JLSISZ5Zt2c2YjFQSPNxBDSoIEZG4Uh0MsWprCeM93kENKggRkbiyfkcpldUhT0+Q20cFISISR77eQa2CEBGRGhZtKqJL+2QGebyDGlQQIiJxZdGmIib270pTXOBaBSEiEieK9laRU7CXiQO7NsnyVBAiInFicV4RABP7qyBERKSGRZuKSEowT28SVJMKQkQkTizaVMSYPp1pl5LYJMtTQYiIxIFAMMQX+cVMHNCtyZapghARiQOrtpZQEQgxcUDT7H8AFYSISFxYtCm8g/qwAU2z/wFUECIicWFRXhEZXdrRO7Vdky1TBSEiEgcWbypq0s1LoIIQEWn2thSXs213hQpCRET+2779DyoIERH5L4s3FdE+JZGRvTo16XJVECIizVzWpl0c2q8LSYlN+5WtghARacYKSytZvW1Pk29eAhWEiEizdv8HOTjnOGNCRpMvWwUhItJM7dhTwROfbuL0CRkMTu/Y5MtXQYiINFP3f5BLIOi49rhhvixfBSEi0gztKKngyU83ccaEDAY2we1Fo1FBiIg0Q/d9kEN1yPGT44b6lkEFISLSzGwvqeCpz/I467AMBnT3Z+0BVBAiIs3OP95fTyjk+IlP+x72UUGIiDQj87MLefLTPC48fAD9urX3NYsKQkSkmdhdFuDnz33BkPQO/HLmSL/jqCBERJoD5xy/fmU5haWV/PW7E5rsvtN1UUGIiDQDry7dyhvLtnH9jGGM65vqdxzA44Iws5lmttbMss3spijjp5vZYjOrNrOza43rb2Zvm9lqM1tlZgO9zCoi4petxeX876srmDigK1cePcTvOF/xrCDMLBG4BzgZGA2cb2aja02WB1wMPB3lLR4H7nTOjQImAzu8yioi4qe/vLOOquoQd517SJNfsbUuSR6+92Qg2zmXC2BmzwKnAav2TeCc2xgZF6o5Y6RIkpxz70SmK/Uwp4iIbzbt3MtLS7bwgyMG+HrOQzReVlUGsLnG6/zIsFgMB4rN7CUzW2Jmd0bWSP6LmV1uZllmllVQUNAIkUVEmtbd72eTlGBc1Yw2Le3jZUFYlGEuxnmTgGnADcAkYDDhTVH//WbOPeCcy3TOZaanpx9oThERX+TtLOOlJVu4YEp/enRu63ecb/CyIPKBfjVe9wW2NmDeJc65XOdcNfAKcFgj5xMR8dXdc9eTmGDNasd0TV4WxEJgmJkNMrMU4DzgtQbM29XM9q0WHEeNfRciIvEub2cZLy3ewgWT+9OzGa49gIcFEfnL/8fAHGA18JxzbqWZzTKzUwHMbJKZ5QPnAPeb2crIvEHCm5feM7PlhDdXPehVVhGRpnbP3GwSEoyrjmmeaw/g7VFMOOdmA7NrDbu5xvOFhDc9RZv3HWC8l/lERPyQX1TGi4vzufDwAc127QF0JrWISJN78MNczOCKowf7HaVOKggRkSZUWFrJsws3c8aEDHqntvM7Tp1UECIiTeiRjzdQFQxxRTM9cqkmFYSISBMpqQjwxIJNnDy2F0PSO/odp14qCBGRJvLkp5vYU1nN1cf4d5/phlBBiIg0gYpAkEc+3sC0YWmMzWgel/OujwpCRKQJPJ+1mcLSKq45Nj7WHkAFISLiufKqIHfPzWbigK5MGdTN7zgxU0GIiHjskfkb2F5SyS9njsQs2nVMmycVhIiIhwpLK7lvXg4njO7J5DhaewAVhIiIp/7+3nrKA0FuOnmk31EaTAUhIuKR3IJSnv4sj/Mn94uL8x5qU0GIiHjk//6zhjZJCVx3/HC/oxwQFYSIiAcW5OxkzsrtXHn0ENI7tfE7zgFRQYiINLKCPZVc9+wSBnRvz6XTBvkd54CpIEREGlF1MMS1zyxhd3mA+743kfYpnt52x1Pxm1xEpBm66511LMjdyZ/OOYTRfTr7HeegaA1CRKSRvLNqO/fOy+H8yf04e2LUm2XGFRWEiEgj2FJczs+fW8rYjM789pQxfsdpFCoIEZGDFAo5bnjuC4Ihxz0XHEbb5ES/IzUKFYSIyEF6ZP4GFuTu5OZTRjOgewe/4zQaFYSIyEFY++Ue7pizlhNG9+TczH5+x2lUKggRkQNUWR3k+n8vpXPbJG4/c1xcXak1FjrMVUTkAP313fWs3lbCwxdlktYxPs+WrovWIEREDsCKLbt54MNcvpvZj+NH9fQ7jidUECIiDRQIhvjFC8vo1iGF//n2KL/jeEabmEREGuihjzawalsJ/7zwMFLbJfsdxzNagxARaYANhXv567vrOGlMT2aO7e13HE+pIEREYuSc41cvLSMlKYFZp431O47nVBAiIjF68rM8Ps3dxf98axQ9O7f1O47nVBAiIjHI3lHKrW+uYtqwNL7bwk6I2x9PC8LMZprZWjPLNrObooyfbmaLzazazM6OMr6zmW0xs7u9zCkiUpeq6hDXPbuEdsmJ/PmcQ0hIaFknxO2PZwVhZonAPcDJwGjgfDMbXWuyPOBi4On9vM3vgQ+8yigiEos/v7OWlVtL+L+zxtOjFWxa2sfLNYjJQLZzLtc5VwU8C5xWcwLn3Ebn3DIgVHtmM5sI9ATe9jCjiEidPsku5IEPczl/cn9OHNPL7zhNysuCyAA213idHxlWLzNLAP4M3FjPdJebWZaZZRUUFBxwUBGRaMqrgvz8+S8Y1L0D//udlntC3P54WRDRNtK5GOe9GpjtnNtc10TOuQecc5nOucz09PQGBxQRqcu/F+axbXcFt585Lq7vLX2gYvoXm9kQIN85V2lmxwDjgcedc8V1zJYP1NzV3xfYGmOuI4BpZnY10BFIMbNS59w3dnSLiHghEAzx4EcbmDSwK1MGd/c7ji9iXYN4EQia2VDgYWAQ+9+xvM9CYJiZDTKzFOA84LVYFuac+55zrr9zbiBwA+EyUjmISJN5belWthSXc9UxQ/yO4ptYCyLknKsGzgD+6pz7KVDnOeaR6X8MzAFWA88551aa2SwzOxXAzCaZWT5wDnC/ma080H+IiEhjCYUc932Qw8henTh2RA+/4/gm1o1qATM7H7gIOCUyrN4rVDnnZgOzaw27ucbzhYQ3PdX1Ho8Cj8aYU0TkoL27ejvZO0r523mHtribADVErGsQlxDeL3Crc26DmQ0CnvQuloiIP5xz3Dsvh37d2vHtcS37Ynz1iWkNwjm3CrgWwMy6Ap2cc3/0MpiIiB8+zd3F0s3F/P70sSQltu6rEcX0rzezeZHLXnQDvgD+ZWZ3eRtNRKRpVQSC3P7WatI6pnDOxDq3frcKsdZjqnOuBDgT+JdzbiIww7tYIiJNyznHjS8sY1n+bv5w+ljaJif6Hcl3sRZEkpn1Bs4F3vAwj4iIL/767npe/2Irv5g5osXfCChWsRbELMKHq+Y45xaa2WBgvXexRESazqtLt/C399Zz9sS+XHV06z3vobZYd1I/Dzxf43UucJZXoUREmsqiTUXc+MIyJg/qxm1njGvVh7XWFutO6r5m9rKZ7TCz7Wb2oplpD46IxLX8ojKueCKL3qlt+eeFE0lJat1HLdUW66fxL8KXyehD+Iqsr0eGiYjEpdLKai59NIvK6hAPXzSJbh1S/I7U7MRaEOnOuX8556ojj0cBXT5VROJSMOS49pklZBeUct/3JjK0R0e/IzVLsRZEoZldaGaJkceFwE4vg4mIeOW22at5f80OfnfqGI4aluZ3nGYr1oL4IeFDXL8EtgFnE778hohIXJmz8kse/ngDFx85kO8fPsDvOM1aTAXhnMtzzp3qnEt3zvVwzp1O+KQ5EZG4sWNPBb96aTljMzrzP99qfXeIa6iD2WX/s0ZLISLiMeccN724nL2V1fzl3EN1xFIMDuYT0sHCIhI3nvl8M++v2cFNJ49kWM9OfseJCwdTELHeX1pExFcbCvfy+zdWMW1YGhcdMdDvOHGjzjOpzWwP0YvAgHaeJBIRaSTOOeas/JJbXl9FSlICd559CAkJ2vgRqzoLwjmn9TARiUu5BaX89rWVfLS+kJG9OnHfhRPpldrW71hxJdZbjoqIxI25a3dw+eNZtE1K5HenjObCwwe0+pv/HAgVhIi0KBWBIL95eQWD0jrw1GWHk96pjd+R4pYKQkRalHvn5bCluJxnL1c5HCytc4lIi5G3s4x/fpDDqYf04fDB3f2OE/dUECLSYsx6YxXJCcavv62zpBuDCkJEWoS5a3bw7urtXHv8MHp21tFKjUEFISJxryIQ5JbXVzI4vQOXTB3kd5wWQzupRSTu3Tcvh407y3ji0sm6xlIj0icpInEtt6CU++aFd0xPG6b7mDUmFYSIxC3nHL95ZQVtkhP4zXe0Y7qxqSBEJG69unQrn+Ts5BczR9Kjk3ZMNzYVhIjEpd1lAf7w5ioO7deF703u73ecFkk7qUUk7pRXBfnFi19QVBbgsR+O1RVaPeLpGoSZzTSztWaWbWY3RRk/3cwWm1m1mZ1dY/ihZrbAzFaa2TIz+66XOUUkfmTvKOX0e+bz9qrt3DRzJGP6pPodqcXybA3CzBKBe4ATgHxgoZm95pxbVWOyPOBi4IZas5cBP3DOrTezPsAiM5vjnCv2Kq+INH+vLt3Cr15aTtvkRB67ZDLTh+uoJS95uYlpMpDtnMsFMLNngdOArwrCObcxMi5Uc0bn3Loaz7ea2Q4gHVBBiLRSLy7K5+fPf8GkgV35x/mH6d4OTcDLgsgANtd4nQ9MaeibmNlkIAXIaaRcIhJnyqqquWPOGib078LTPzqcZN3boUl4+SlH22vUoPtYm1lv4AngEudcKMr4y80sy8yyCgoKDjCmiDR3j3y8ge0llfz6W6NUDk3Iy086H+hX43VfYGusM5tZZ+BN4DfOuU+jTeOce8A5l+mcy0xP17ZIkZaosLSSf36Qy0ljepI5sJvfcVoVLwtiITDMzAaZWQpwHvBaLDNGpn8ZeNw597yHGUWkmfvHe+spDwT5xcyRfkdpdTwrCOdcNfBjYA6wGnjOObfSzGaZ2akAZjbJzPKBc4D7zWxlZPZzgenAxWa2NPI41KusItI8bSjcy1Of5XH+5H4MSe/od5xWx9MT5Zxzs4HZtYbdXOP5QsKbnmrP9yTwpJfZRKR5c85xx3/WkJKUwHXHD/c7TqukvT0i0uzsrazmZ899wVsrvuTKo4fo3tI+0aU2RKRZWbd9D1c9uYjcwr38dMZwrjl2qN+RWi0VhIg0G28t38ZPn1tKxzbJPHXpFI4cmuZ3pFZNBSEizcKeigA3vbScET078eBFmbp8dzOgfRAi0iw89slGdpcH+MPp41QOzYQKQkR8V1IR4MGPNjBjVA/G9dXVWZsLFYSI+O6x+eG1Bx3O2ryoIETEVyUVAR76WGsPzZEKQkR89Whk7eH6GVp7aG5UECLim5KKAA99lMuMUT0Zm6G1h+ZGBSEivqisDvLTZ5dSUlHN9TOG+R1HotB5ECLS5CoCQa54YhEfrCvgD6eP1dpDM6WCEJEmVREI8qPHs/g4u5A/njmO8yb39zuS7IcKQkSaTHUwxGWPZTE/p5A7zhrPOZn96p9JfKN9EIQvKxwMNehuqCJyAB6Zv+GrNQeVQ/PX6gsiv6iMo/5vLm8u3+Z3FJEWbfOuMv7yznpmjOrBuSqHuNDqC6J3ajvKA0HeW73d7ygiLZZzjptfXYEZ3HLaWMzM70gSg1ZfEIkJxrEjejBvbQHVwZDfcURapDeXb2Pu2gJ+dsJwMrq08zuOxKjVFwTAjFE92F0eIGtTkd9RRFqc3eUBbnl9FWMzOnPxkQP9jiMNoIIApg1PJyUxgXdXaTOTSGNyzvH7N1axs7SS288YT1KivnLiif5vAR3bJDFlcDfeW7PD7ygiLcp9H+TwwqJ8rjl2qC7EF4dUEBEzRvVkQ+FecgpK/Y4i0iK8unQLd/xnLace0oef6kJ8cUkFEXH8qB4AOppJpBEsyNnJDc9/wZRB3bjznPEkJOiopXikgojo27U9I3t14t3V2swkcjCyNu7i8ieyGNi9Aw98P5M2SYl+R5IDpIKoYcaonizaVERxWZXfUUTiTnlVkFmvr+Kc+xeQ2i6Zf10yidT2yX7HkoOggqjh+FE9CIYc89YW+B1FJK4syNnJzL99yCPzN3DhlAH85/rp9O3a3u9YcpB0sb4aDunbhbSObXh39XZOn5DhdxyRZm/X3ipun72a5xflM6B7e569/HAOH9zd71jSSFQQNSQkGMeNTOetFV8SCIZI1jHbIlE553hhUT63zV7Nnopqrjx6CNcdP4x2Kdrf0JKoIGo5dkQPnsvKZ+nmYiYN7OZ3HJFmxTnHR+sL+cu761iSV0zmgK7cesY4RvTq5Hc08YAKopYjhnTHDOZnF6ogRCL2FcNf313H4rxi+qS25Y6zxnP2xL46hLUFU0HU0qV9CmP7pPJJ9k6un+F3GhH/7S4LcNNLy3hrxZf0SW3LrWeM5ZyJ/UhJ0ibYlk4FEcWRQ7vzyMcbKKuqpn2KPiJpvRZu3MV1zyxhx55KfjFzBJcdNVjF0Ip4+n/azGaa2Vozyzazm6KMn25mi82s2szOrjXuIjNbH3lc5GXO2qYOSSMQdHy+YVdTLlbEd4FgiA2Fe5m7Zge3z17Nd+9fQHJSAi9edSRXHzNU5dDKePbnsZklAvcAJwD5wEIze805t6rGZHnAxcANtebtBvwWyAQcsCgyb5Ncj3vSwG6kJCbwSc5OjhnRoykWKdLkNu8q48P1BeQW7GVD4V42Fu4lb1cZ1TVuv3vmhAxmnT6Wjm20Jt0aefl/fTKQ7ZzLBTCzZ4HTgK8Kwjm3MTKu9p16TgLecc7tiox/B5gJPONh3q+0S0lkQv8uzM8ubIrFiTSZ4rIq3ly+jVeWbGHhxvDfW22TExiU1pFRvTtz8rheDOzegcHpHRiU1pFuHVJ8Tix+8rIgMoDNNV7nA1MOYt5vnLlmZpcDlwP079//wFLux9Shafzl3XUU7a2iq35JJM4553jyszxue3M15YEgQ3t05MaTRvDtcb3p3629jkSSqLwsiGg/cS7KsAOe1zn3APAAQGZmZqzvHZOpQ7tz1zuwIHcn3xrXuzHfWqRJ7Sip4BcvLmPe2gKmDUvjlzNHMqZPZ90XWurlZUHkA/1qvO4LbG3AvMfUmndeo6SK0fi+XeiQksj87EIVhMSlQDDEK0u2cOvs1VQEgsw6bQzfP3yAikFi5mVBLASGmdkgYAtwHnBBjPPOAW4zs66R1ycCv2r8iPuXnJjAlMHd+SRnZ1MuVuSglVcFeS5rMw98mMuW4nIO6deFu849hCHpHf2OJnHGs4JwzlWb2Y8Jf9knAo8451aa2Swgyzn3mplNAl4GugKnmNktzrkxzrldZvZ7wiUDMGvfDuumNHVoGu+v2cGW4nIyurRr6sWLxGxrcTmf5u5kQc5O3luzg117q5g4oCuzThvDsSN6aB+DHBBPj11zzs0GZtcadnON5wsJbz6KNu8jwCNe5qvP1KHhq1LOzy7k3Mx+9Uwt4r2CPZU8/Vkey7fsZk9FgJKKaor2VvFlSQUAXdonM3VIGhdPHahLxchB08HNdRjRsxNpHVP4eL0KQvy1bvseHv5oAy8v3UJVdYiRvTrRuV0yGV3aMqpXJ8ZkpHLE4O6M7NVJawvSaFQQdTAzThrTi+eyNpNfNKLeG6BsKS6nIhDUtl5pNJt3lfHH/6zhzWXbaJucwLmZfblk6iD9jEmTUEHU45pjh/J8Vj5/f289d5x9yH6n21layRn3zGfHnkpmjOrBVccMZeKArvudXqQuu8sD3Ds3m3/N30hCAvzkuKFcMnWQTlyTJqWCqEefLu248PABPLZgI1ccPSTqX27OOW58YRnF5QEuO2oQLyzO56z7PmHKoG785tujGdc3temDS9xYnFfEIx9vYNW2EiqqgpQFguytrKY65DjrsL7ccOIIeqW29TumtELmXKOeX+abzMxMl5WV5cl7F5ZWMv2OuRw3sgd3X3DYN8Y//PEGfv/GKm45dQwXHTmQsqpqnvl8M//8IIfisipuPCl8FUxtG5Z9QiHHnJVf8uBHuSzOK6Zz2ySOGpZGh5Qk2qUk0j4lie+M783YDP1xId4ys0XOucxo47QGEYO0jm24ZOpA7pmbw9XHlDC6T+evxi3P380f31rNCaN78oMjBgDQPiWJS48axJkTMrjppWXcNnsNH64r5K5zD6FHZ/0l2NrtrazmumeX8u7q7fTv1p7fnTKaczL70UEXxJNmRmsQMdpdFmDaHe8zeVA3HrpoElXVIXILS7nyiUVUVoeYfe20qNdscs7xzOebmfXGSpISEjh8cHeOGNJdR5y0Utt2l3Ppo1ms+bKE33x7NBcdOZBE/QyIj7QG0QhS2ydzxdFDuHPOWo778zzydoYvi5yYYDx92ZT9XtDPzLhgSn8mD+rKQx9tYEHuTt5dvR2Ao4am8eglk0hK1DX2W4LtJRVkbSyia/tkenRuQ3qntnRISaQ65AgEQ6zbXspVTy6irCrIwxdP4lhdSl6aORVEA1x85EAWbtxFUkICJ4/txfCenTi0XxcGdO9Q77xDe3Tij2eNB8KHw76yZAt3zlnL39/P5mcnDPc6uhD+Al+0qYjFm4rYXFTGgO4dGJrekSE9wpe1rqoOUVkdpKo6xJ7KavZUVFNSHqA6GCKja3v6dws/2qUkfvWeoZDjo+xCnv5sE++u3kEwVPcaeUaXdrx41RRG9Ork9T9X5KCpIBqgQ5skHr1k8kG/T0aXdlxz7FByCkq5+/31HDmkO4cP7t4ICaW2quoQLyzK58GPctlQuBeAlKQE+nZtx9y1BVRV174VSf06pCSSlJhAcqIRCDp2lwfo1iGFy44axMnjelNWVU3Bnkp2lFRSHgiSlGgkJyTQNjmBk8f1Jq1jm8b+Z4p4QgXho9+fNpYlecVc/+xS3rouvA9jR0kF/3g/mw/WFTA4vQNj+6Qypk9njhySRmr7ZL8jx42q6hDPL9rMvXNz2FJczqH9uvC/3xnNYf27MKZPKilJCQRDjs27ysjeUcqeygApiYmkJCWQkpRAxzZJpLZLonPbZBISjPyicjbt3EvezjKKI2sV1SFHyDmOGJLGSWN60iYpsf5gInFEO6l9tmLLbs64dz7ThqUzrEdHHluwkeqgY/rwdLYUlZNdUEow5Mjo0o6XrzmSHp10FFR9Nu8q47LHsli7fQ+H9e/CdTOGM31Ymi5zLRJFXTupVRDNwL7zKMzg9EMzuH7GsK/2a1QEgnyau5OrnlzMkB4d+PflR+hwyDosyNnJ1U8tIuTgT+ccwoxRPVQMInVQQTRzzjleWJTPIf26MLxn9J2X763ezo8ez+Lo4ek8+IPMFnnkU8GeSj7fsIvi8ioqAyEqq0OUV1VTXB6guCxAcXmA1HbJHDmkO1OHpNG/+9fXxgqGHM98nsfvXlvJgO7teeiiSQxKq//gAZHWTgXRQjz12SZ+/fIKzp/cn9vOGHtQfxmXVVWzaFMRKYkJZA7sdtDH4pdXBVnzZQmrtpWwamsJ+UXl7Npbxc7SSorLAwzo3oHD+ndhQv+ujOzViYpA8KsjhVZtLeHDdQWs2lbyjfc1g85tk+naPpnUdsls213Bjj2VAPRJbUtyUgLFZQFKKgI4B8eOSOdv50+gc1vtrxGJhc6DaCG+N2UAW4rKuXdeDrv2VvKrk0cxsAF/JZdUBHjk4w18kr2TJZuLCATDfxz07NyGU8b34bRDMxibUfe9ineXBXjys0089ekmisoChFx4R+2+9wLo1DaJQWkdSOuYwvCenejcLonsHaW8tnQrT32W9433TE40Jg7oyo0njWDq0DR6p7alTVICbZISaZOU8F8nEzrnyCkoZX72ThZu3EVigtGlXbg8+nfvwBkTMnTimUgj0RpEnAmFHPd9kMM9c0epGWkAAAlASURBVLMJBEN8//CBXHv8ULq0r/8qn9c8tZjZK7YxLiOVI4Z058ghaeypCPDKkq18sG4HgaBj8sBu/PzE4Uypddjt5l1lPL5gI09/lsfeqiDTh6czslcnzCDBjHbJiYzo1YnRvTvTt2u7qCUTCjmyC0rJLSilfUoSHdsm0bltEr1T22m/iohPtImpBdpRUsFd76zjuazNdEhJ4pRD+3BuZj8O6Zsa9cv53VXbuezxLG44cTg/Pm7YN8YXl1Xx8pIt3Dsvh4I9lUwblsY5mf1YmlfMB+t2kFOwlwSDUw7pwxXTh/zX9ahEJH6pIFqwNV+W8MAHucxesY2KQIhhPTpy5dFDOGvi13dyLa2s5sS7PqBj2yTe+Mk0UpL2v4O7vCrIE59u5L55ORSVBUhJSmDKoG4cM6IHJ43pWe9Nk0QkvqggWoE9FQHeWLbtq/sVnz+5P7ecOoaUpARueX0lj36ykReuPDLmmxjtqQiwetsexmZ0pn2KNv+ItFTaSd0KdGqbzPmT+3NuZj/+9PZa7puXw7rte7jy6CE8+slGLpwyoEF3uOvUNpnJg3TTe5HWrOUdTN/KJSYYv5w5krsvmMCqrSX86PEsenRqw40zR/gdTUTijNYgWqjvjO/DkPSO/O61lVx97FCdFyAiDaaCaMFG9e7Mv684wu8YIhKntIlJRESiUkGIiEhUKggREYlKBSEiIlGpIEREJCoVhIiIRKWCEBGRqFQQIiISVYu5WJ+ZFQCbgFRgd2Rwzec1X0ebJg0obOBia79/rNPUN6y+5zWHeZF7f+MbK/eBZK4rV33j9/dzEO21ctefq77xB5K7sX4n68pV1/hYhnmZ28/vki7OufSoS3TOtagH8EC05zVfR5sGyDqYZTVkmvqG1fe81rBGz72/8Y2V+0AyN2buul4rtz+5G+t38kBzxzLMy9zN4bsk2qMlbmJ6fT/Pa76ua5oDXVZDpqlvWH3PDyZzLPPvb3xLyV3Xa+Xe//JiHX8guRvrdzKW+Q/kd7L261bxXdJiNjEdLDPLcvu5JnpzFo+54zEzKHdTU27/tcQ1iAP1gN8BDlA85o7HzKDcTU25faY1CBERiUprECIiEpUKQkREolJBiIhIVCqIepjZNDP7p5k9ZGaf+J0nVmaWYGa3mtk/zOwiv/PEysyOMbOPIp/5MX7naQgz62Bmi8zsO35niZWZjYp81i+Y2VV+54mVmZ1uZg+a2atmdqLfeWJhZoPN7GEze8HvLLFq0QVhZo+Y2Q4zW1Fr+EwzW2tm2WZ2U13v4Zz7yDl3JfAG8JiXeWvkO+jcwGlABhAA8r3KWlMj5XZAKdCW+MoN8EvgOW9SflMj/Xyvjvx8nws0yaGZjZT7Fefcj4CLge96GHdftsbInOucu9TbpI3sQM5UjJcHMB04DFhRY1gikAMMBlKAL4DRwDjCJVDz0aPGfM8BneMlN3ATcEVk3hfiKHdCZL6ewFNxlHsGcB7hL6zvxEvuyDynAp8AF8RT7sh8fwYOi7PMTfL72BiPJFow59yHZjaw1uDJQLZzLhfAzJ4FTnPO3Q5E3TRgZv2B3c65Eg/jfqUxcptZPlAVeRn0Lu3XGuvzjigC2niRs7ZG+ryPBToQ/oIoN7PZzrlQc88deZ/XgNfM7E3gae8Sf7W8xvi8Dfgj8JZzbrG3iRv9ZztutOiC2I8MYHON1/nAlHrmuRT4l2eJYtPQ3C8B/zCzacCHXgarR4Nym9mZwElAF+Bub6PVqUG5nXO/BjCzi4FCr8uhDg39vI8BziRcxrM9TVa3hv58/4TwWluqmQ11zv3Ty3D70dDPujtwKzDBzH4VKZJmrTUWhEUZVufZgs6533qUpSEalNs5V0a42PzW0NwvES43vzX45wTAOfdo40dpkIZ+3vOAeV6FaYCG5v478Hfv4sSkoZl3Ald6F6fxteid1PuRD/Sr8bovsNWnLA2h3E1LuZtWPOaOx8wN0hoLYiEwzMwGmVkK4R2Lr/mcKRbK3bSUu2nFY+54zNwwfu8l9/IBPANs4+tDPS+NDP8WsI7wEQi/9junciu3cjfv3PGYuTEeulifiIhE1Ro3MYmISAxUECIiEpUKQkREolJBiIhIVCoIERGJSgUhIiJRqSCkRTOz0iZe3kNmNrqR3itoZkvNbIWZvW5mXeqZvouZXd0YyxYBdB6EtGxmVuqc69iI75fknKturPerZ1lfZTezx4B1zrlb65h+IPCGc25sU+STlk9rENLqmFm6mb1oZgsjj6mR4ZPN7BMzWxL574jI8IvN7Hkzex1428J3vZtn4buwrTGzpyKXnyYyPDPyvNTCd/X7wsw+NbOekeFDIq8XmtmsGNdyFhC+eihm1tHM3jOzxWa23MxOi0zzR2BIZK3jzsi0N0aWs8zMbmnEj1FaARWEtEZ/A/7inJsEnAU8FBm+BpjunJsA3AzcVmOeI4CLnHPHRV5PAK4nfP+HwcDUKMvpAHzqnDuE8CXXf1Rj+X+LLL/ei7uZWSJwPF9f56cCOMM5dxhwLPDnSEHdBOQ45w51zt1o4VtxDiN834JDgYlmNr2+5Yns0xov9y0yAxgd+aMfoLOZdQJSgcfMbBjhyzYn15jnHefcrhqvP3fO5QOY2VJgIPBxreVUEb6bGMAi4ITI8yOA0yPPnwb+tJ+c7Wq89yLgnchwA26LfNmHCK9Z9Iwy/4mRx5LI646EC8PP+4NIHFFBSGuUABzhnCuvOdDM/gHMdc6dEdmeP6/G6L213qOyxvMg0X+XAu7rnXz7m6Yu5c65Q80slXDRXEP4HgjfA9KBic65gJltJHwP79oMuN05d38DlysCaBOTtE5vAz/e98LMDo08TQW2RJ5f7OHyPyW8aQvCl4iuk3NuN3AtcIOZJRPOuSNSDscCAyKT7gE61Zh1DvBDM9u3ozvDzHo00r9BWgEVhLR07c0sv8bjZ4S/bDMjO25X8fVdvu4Abjez+YRvSO+V64GfmdnnQG9gd30zOOeWAF8QLpSnCOfPIrw2sSYyzU5gfuSw2Dudc28T3oS1wMyWAy/w3wUiUicd5irSxMysPeHNR87MzgPOd86dVt98Ik1N+yBEmt5E4O7IkUfFwA99ziMSldYgREQkKu2DEBGRqFQQIiISlQpCRESiUkGIiEhUKggREYlKBSEiIlH9P6VySHjrg7ERAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn.lr_find()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy_multi</th>\n",
       "      <th>fbeta_score</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0.097059</td>\n",
       "      <td>0.090399</td>\n",
       "      <td>0.955622</td>\n",
       "      <td>0.921118</td>\n",
       "      <td>01:01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.094195</td>\n",
       "      <td>0.090244</td>\n",
       "      <td>0.954613</td>\n",
       "      <td>0.918630</td>\n",
       "      <td>01:01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0.089849</td>\n",
       "      <td>0.086629</td>\n",
       "      <td>0.954838</td>\n",
       "      <td>0.923924</td>\n",
       "      <td>01:01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>0.079172</td>\n",
       "      <td>0.084119</td>\n",
       "      <td>0.959031</td>\n",
       "      <td>0.926660</td>\n",
       "      <td>01:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>0.076948</td>\n",
       "      <td>0.084122</td>\n",
       "      <td>0.958355</td>\n",
       "      <td>0.926558</td>\n",
       "      <td>01:02</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn.fit_one_cycle(5, slice(1e-5, lr/5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "learn.save('stage-2-rn50')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tfms = aug_transforms(flip_vert=True, max_lighting=0.1, max_zoom=1.05, max_warp=0., size=256)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.random.seed(42)\n",
    "dls = planet.dataloaders(df, bs=64, path=path, batch_tfms=tfms+[Normalize.from_stats(*imagenet_stats)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "learn.dls = dls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "learn.freeze()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXyU1b3H8c8ve0JCEgh7EnYQUBAIIO77Wpe6r3WrS1vrbe1mb3v1Xlt7vW1trba2VVtrxR214laUqnVhkYQlbAoBIQlhSwKBJCQhmXP/mEmchAESyJOZCd/365WXM888M8+PMZnvnHOe8xxzziEiItJWTLgLEBGRyKSAEBGRkBQQIiISkgJCRERCUkCIiEhICggREQkpLtwFdJasrCw3ZMiQcJchIhJVCgoKyp1zfUI91m0CYsiQIeTn54e7DBGRqGJmG/b1mLqYREQkJAWEiIiEpIAQEZGQFBAiIhKSAkJEREJSQIiISEgKCBGRKLZwfSUFGyo9eW0FhIhIFHtozmp+9sYqT15bASEiEsU2VNQyuHeKJ6+tgBARiVINjT7KduxmcC8FhIiIBNm4Yzc+B7m9e3jy+goIEZEotaGiBkBdTCIi0lpJZS0AuepiEhGRYBsqakmKj6FvWqInr6+AEBGJUhsqa8ntlYKZefL6CggRkShVXFFLbi9vBqhBASEiEpWccxRXejcHAhQQIiJRaduuenbvafJsgBoUECIiUam4+QwmtSBERCTYhgp/QHg1ixoUECIiUWlDZS0xBtmZCggREQlSXFHDgPRkEuK8+xhXQIiIRKHmORBeUkCIiEShEo9PcQUFhIhI1Kmub6S8usHTM5hAASEiEnWKW85g8m4WNSggRESiTnGlt5f5bqaAEBGJMs1zIHKieZDazM42s8/NrMjM7g7x+IlmtsjMGs3s0qDtR5vZPDNbYWaFZnaFl3WKiESTDZW1ZKTEk54c7+lxPAsIM4sF/gCcA4wFrjKzsW12KwZuAJ5ts70W+JpzbhxwNvCQmWV4VauISDQpqaz1dAZ1My9bEFOBIufcOudcA/A8cGHwDs659c65QsDXZvtq59yawO0yYCvQx8NaRUSixoaKWs/WoQ7mZUAMAkqC7pcGtnWImU0FEoC1nVSXiEjU2tPkY+OO3eT2Svb8WF4GRKgljlyHXsBsAPA0cKNzzhfi8VvNLN/M8rdt23aQZYqIRI+yHbtp8jnPT3EFbwOiFMgJup8NlLX3yWbWE3gT+Klzbn6ofZxzjznn8pxzeX36qAdKRLq/5jOYvJ4kB94GxEJgpJkNNbME4EpgVnueGNj/VeDvzrmXPKxRRCSqrNtWDcCQaB6DcM41AncAs4FVwIvOuRVmdp+ZXQBgZlPMrBS4DPizma0IPP1y4ETgBjNbEvg52qtaRUSiReHGKrJSE+nXM9HzY8V5+eLOubeAt9psuyfo9kL8XU9tnzcDmOFlbSIi0aiwtIoJ2emYhRrm7VyaSS0iEiWq6xtZu62ao7LTu+R4CggRkSixfGMVzsGE7K6ZN6yAEBGJEoWlOwDUghARkdYKS6sYlJFMVqr3A9SggBARiRqFpVWM76LWAyggRESiwo7aBoora7usewkUECIiUaGwtArougFqUECIiESF5gHqIwepBSEiIkEKS6sYmtXD80WCgikgRESiQFcPUIMCQkQk4m3dWcfmnXWM78LxB1BAiIhEvOYBarUgRESklcLSHcQYjBvYs0uPq4AQEYlwS0urGNk3jZQETy/AvRcFhIhIBHPOsWxj1w9QgwJCRCSirSjbSWVNA5MHZ3b5sRUQIiIR7PXCMuJijLPG9e/yYysgREQilM/neGPpJk4YmUVmj4QuP74CQkQkQi0q3s7GHbu54OiBYTm+AkJEJELNWlpGYlwMZ4zt+u4lUECIiESkxiYfby3bxOlj+pGa2LWntzZTQIiIRKC5aysor27g/Anh6V4CBYSISESatbSMtMQ4Th7dJ2w1KCBERCJM3Z4mZi/fzFlH9icpPjZsdSggREQizAefb2NXfSMXhLF7CRQQIiIR581lm+jdI4Fjh/cOax0KCBGRCOKc45Oick4a3Ye42PB+RCsgREQiyJqt1VTWNDB9WHhbD6CAEBGJKPPWVgBwjAJCRESCzV9XwaCMZHJ6pYS7FAWEiEik8PkcC76ojIjWAyggREQiRvP4wzHDeoW7FEABISISMeavi5zxB1BAiIhEjPnrKsjOjIzxB1BAiIhEhEgbfwAFhIhIRPhy/EEBISIiQZrHH6YNjYwBalBAiIhEhEgbfwAFhIhI2EXi+AN4HBBmdraZfW5mRWZ2d4jHTzSzRWbWaGaXtnnsn2a2w8ze8LJGEZFwi8TxB/AwIMwsFvgDcA4wFrjKzMa22a0YuAF4NsRL/Aq4zqv6REQixXufbQVgepgv792Wly2IqUCRc26dc64BeB64MHgH59x651wh4Gv7ZOfcv4BdHtYnIhJ2zjleXlRK3uBMBmUkh7ucVrwMiEFASdD90sC2TmNmt5pZvpnlb9u2rTNfWkSkSywp2UHR1mounZwd7lL24mVAWIhtrjMP4Jx7zDmX55zL69MnfAt7i4gcrJkFpSTFx3De+AHhLmUvXgZEKZATdD8bKPPweCIiUaVuTxOzlpZxzpEDSEuKD3c5e/EyIBYCI81sqJklAFcCszw8nohIVHln5RZ21TVyWQR2L4GHAeGcawTuAGYDq4AXnXMrzOw+M7sAwMymmFkpcBnwZzNb0fx8M/sIeAk4zcxKzewsr2oVEQmHl/JLGJSRHHGntzaL8/LFnXNvAW+12XZP0O2F+LueQj33BC9rExEJp01Vu/m4qJxvnzqSmJhQQ7bhp5nUHvH5HL95dzWvLdlIY9NeZ/GKyGHulUUbcQ4unRSZ3UvgcQvicFZcWcvD/1oDwC//+Tm3nDCUy6fkkJKgt1zkcOecY2ZBKdOG9iK3d+Rce6kttSA8UlxZC8Cdp45gQHoS//36Sk799b/ZUdsQ5spEJNze/3wrX5TXcOXUnAPvHEYKCI+UbPcHxJVTc5n5jWN58sYpbN5Zx0v5pWGuTETCyTnH798rYlBGMl8ZPzDc5eyXAsIjJZW7iY81+vVMAuCU0X2ZOqQXT8/fgM/XqfMFRSSKLPiikkXFO7j9pGHEx0b2R3BkVxfFSrbXMigjmdigsxOumz6Y4spa/r1alwUROVz94f0islITuSwvsruXQAHhmdLK2r0W/jhrXH/6piXy1Lz1YalJRMJrackOPlpTzi0nDCUpPjbc5RyQAsIjJdt37xUQCXExXD0tlw8+38b68powVSYi4fKH94vomRTHNccMDncp7aKA8EB1fSOVNQ3kZO59+trVU3OJizFmzN8QhspEJFxWb9nFOyu3cMNxQ0lNjI7T3RUQHigJnOKa02vva7v37ZnEOUcN4MX8EmobGru6NBEJgy076/jJq8tISYjlxmOHhLucdlNAeKAlIEK0IAC+Nn0wO+saeW2JLm4r0p0553i5oJQzfvNvlm2s4r4LjySzR0K4y2q36GjnRJmS7bsB9hqDaJY3OJMxA3oyY/4Grpqa25WliUgX+aK8hvvfXMmcVVvJG5zJry6bwNCsHuEuq0MUEB4oqawlNTGOzJTQ13c3M66amsM9r61gRVkV4wamd3GFIuIF5xwL12/n8Y/WMWfVFhJiY/jpeWO48bihrU55jxbt6mIys+Fmlhi4fbKZ3WlmGd6WFpnaM25Qur2W7MxkzPb9C3HBhIEkxMZoZrVIN9HQ6OPqxxdw+Z/nsXB9JXecMoKPfnQKXz9hWFSGA7R/DOJloMnMRgB/AYYCz3pWVYQq2LCdife9y7MLive7X3GIORBtZaQkcOa4fvxjyUbqG5s6s0wRCYPZKzYzb10F3z9zFPPuPo3vnTmavmlJ4S7rkLQ3IHyBBYC+CjzknPsuEHkLqHpoR20Ddz63mPpGH3/5eB3Ohb5chnOOksrd+xygDnZ5Xg47avcwZ+XWzi5XRLrYjPkbyOmVzDdPHkFyQuRPgmuP9gbEHjO7CrgeeCOwLfIWUPWIc44fzCxk6646rp8+mLXbavj0i8qQ+1bUNLB7T1PIU1zbOm5EFgPTk3gxv6SzSxaRLlS0dRcLvqjk6qmDI3bxn4PR3oC4EZgO3O+c+8LMhgIzvCsrsvxt7nreXbmFH519BHefM4a0pDie/TR0N9OBTnENFhtjXDo5mw/XbKNsx+5OrVlEus4zC4qJjzUuy4vcxX8ORrsCwjm30jl3p3PuOTPLBNKccw94XFtEKCzdwS/eWsXpY/py8/FDSU6I5ZJJ2by9bDOVNXuv7dB8imt7FwG5dHIOzsErizRYLRKNdjc08XJBKWcfOYCs1MRwl9Op2nsW0wdm1tPMegFLgSfN7DfelhYZfvnPz+nVI4FfXTqh5aykq6fl0tDkY2bB3l1DzS2I7MwDdzGBP0iOGdaLlwpK9zmuISKR643CMnbWNXLNtO43p6m9XUzpzrmdwMXAk865ycDp3pUVOVZt2skpo/u2mv04ql8aeYMzee7Tkr0+1Esqa8lKTejQ0qKX5+WwoaKWBfsY1xCRyDVjQTEj+qYybWivcJfS6dobEHFmNgC4nC8Hqbu97TUNVNQ0MLxP6l6PXT0tly/Ka5i3tqLV9pLttWS3Y/wh2NlH9icuxvhQ60SIRJXlG6tYWrKDa6bl7nfeU7Rqb0DcB8wG1jrnFprZMGCNd2VFhrXbqgEY0XfvgDj3qAGkJ8fzTJvB6pLKvS/zfSApCXGMHdiTRcXbD75YEelyM+ZvICk+hosndq/B6WbtHaR+yTk33jn3jcD9dc65S7wtLfyKtvoDIlQLIik+lksnZ/POis1sqPCv7dDkc5Tt2E1OO8cfgk3MyaCwtIrGJt+hFS0iXWJ9eQ0zC0q5dHI26fu4rE60a+8gdbaZvWpmW81si5m9bGbdMzKDFG2tJjEuhkH7+MC/5YRhJMXF8uNXluGcY1PVbhp9jtwOtiAAJuZmUtvQxOot1Yda9j79c/lm7npxCXV79p65Xbenia8/lc/C9RoHEWmPB99dTXxsDHeeOjLcpXimvV1MTwKzgIHAIOD1wLZube22aob1Sd3ndVT6pydx97lHMHdtBS/ml1BSuf+ruO7PpNxMABaXHFo3U9HWago2VLYaPPf5HL9553Nun1HAK4s2MnvF5r2eN3vFZuas2sJ9r6/U2VQiB7CstIrXl5Zx8/FD6dszui+nsT/tDYg+zrknnXONgZ+/AX08rCsiFG2rZnif/V+e96opuUwd2oufv7mKgg3+b9/tmSTXVk6vZHr3SGBx8Y6DqhX8FxK8+vH5XPLHeZz78Me8uLCE8up6bn26gIffK+KyydlkZyaHnLn9Yn4JcTHGso1VzFmlS3+I7M8vZ39GZko8t540LNyleKq9AVFuZteaWWzg51qg4oDPimJ1e5oo3b475AB1sJgY44GLj6K+0cfv/rWGGIMBGR3/RmFmTMzNYPEhDFQ/9uE6tu6q59unjsDnc/zw5UKm3D+H9z/fyn+fP5ZfXjqeyybn8ElRRct8DfCfmvtJUQXfPHk4Q3qn8Nt3Vx+2rQjnHG8WbmLNll0H3Leh0cczCzbw2IdreXZBMbOWljF/XYXGkbq5j9eU89Gacr51ygh6JnXPsYdm7T1Z/ybg98BvAQfMxX/5jW5r3bYanAt9BlNbw/qk8p3TR/LLf37OoIxk4mMPbqG+ibmZzFm1laraPR0e9Nqys44//3sd5x7Vn++dOZq7zhjFvHUVvLpoI1+dNIhjh2cBcGleNg/9azUv5Zdw15mjAXipoBQzuGJqLoN79+B7Ly1l9ootnH1k/4P6d0SzxSU7+NazizCDs8f1545TR4Rcr6OkspY7nlvM0pK9W3z9eiZy8aRsLpuczbAQJzhI9PL5HP/3z88YlJHMtccMDnc5nmtXQDjnioELgreZ2XeAh7woKhIUbdv3GUyh3HLCMP65fDP9DqE/cmKOf4mNxSXbOXl03w4999ezP6fR5+NHZx8B+Fskxw7PagmGZoMykjlhZB9eKijlP04fBcDM/BKOH5HFoIxkLjx6IL9/v4iH5qzmzLH9utWFx9pjxvwN9EiI5YbjhvD3uRt4e/lmThrVh1NG92HasN6M7pfG28s3c/fLhWDw6DWTOGFkFjX1TVTX72HNlmpeXlTKYx+u448frGXy4EwunZzNeeMH7PfbZmOTD5+DhDitAhypnHP87l9rWLaxigcvm0BSfPe4Yuv+HMqKcnfRjQNi7dZqzGj3EoHxsTG8eNt0Yg5hssz4nAxiDBYX7+hQQKwoq2LmolJuPm4og3sfuN4r8nL41rOL+GjNNmJjjLKqOv7zvDEAxMXGcOdpI/juC0uZvWIz5xx1+FzVfXtNA28UbuLyvGx+cNYR3HricJ6au54XFpbw78AkxrTEOHbVN3J0TgaPXDWx5YSEtKR4IIkRfdM456gBbN1ZxyuLNzKzoJQfv7KM/561gjPH9Wdk31R6JMaRmhhLkw9Wbqpi+cadrNq0k4YmH33TEhmUkUxurxS+fsIwjhyk1QYjQd2eJn44s5BZS8u46OiBXDRxULhL6hKHEhDd+qtl0bZqcjJTOvQt4VC/UaQmxjGqXxqLQ3Rb7ItzjvvfXEV6cjzfbufpdqeP7UtmSjwv5pcQY0ZGSjxnjO3X8vgFEwbxyHtF/Obd1UzIyWBgRvvmddQ3NrF1Zz2bquoYkpUS0Yul1O1p2uv/18uLSmlo9HHNNH/XQXpyPHeeNpI7TxtJ6fZaFqyr5NMvKsnOTOb2k4fvtyuxb88kbj9pOLedOIzC0ipmFpTy5rJNvL60rNV+aYlxjBvUk+uOGUxqUhwbt++mdPtuPli9jXdXbuHP1+Vx/MisfRxFusLmqjpufTqfZRur+OHZo/nGScO75azpUA4lILr1KObardXtGn/obBNzM3mzsAyfz7V07zRP2AtVz4drypm7toJ7zx/b7nGLxLhYLpo4iBnzN2AYV0/LJTHuyw/L2BjjP88Zw+0zCjjpV+9z0dGDuO2k4ft8P2av2Mx//WM5W3fVt2zLTInnuVuP4Yj+Pdv9b+8qD77zOY9/tI5nvj6NyYP918/x+RzPLChm8uBMxgzYu+bszBSyJ6dwyeSOTf8xMybkZDAhJ4OfXXQke5p81NY3Ud3QiM/nGJSRHLIbb8vOOq7/66fc+LdPefDyo7lgwsCQr19d38jSkh3kZKa0+wrC0n4ry3Zy498+pbqukceuy2v1RepwsN+AMLNdhA4CAzo+XThKNPkc68prOHFU15/JOzE3g+c+LWZdeQ0j+qayZssuvvroXDJS4vnwB6fs9WHy7IINZKUmtnzrba8rpuTw5CfrAcfleTl7PX762H588IOTeeKjL3h+YTEzF5Vy5ZRcfnbhOOKCvjl/tnkn33l+CUOyenDtMYPpn55Ez6R47p21nGufWMDztx7DiL5pB/NWeGJmQSmPvFdEQmwM35ixiDe+fTx9eyYxb10FX5TXcOdpIzw9fnxsDOkpMQcM8349k3jhtunc8vd87nxuMeu2VXNE/574nMPnHEVbq/mkqJzFxTto9Pn/RHN6JXPc8CxOGtWHM8f1j9p1kCPF3KJybn26gLSkOF7+5rER+WXHa/sNCOdc5Pxld6HS7bU0NPoOOAfCC5NyAwPVxdvJSk3glr/nU9/YROn2Ruatq+C4EV92N2yvaeC9z7Zy/fQhHR7cPKJ/TyYPzqTJ5xg7MPQvfnZmCv99wTjuOHUEj76/lr9+8gVVuxv43ZUTiY+Noap2D7cF/oCeunFKqwlDI/ulcsWf53P14wt44bbp7R7L8dLC9ZX8+JVCjhvRmx+fM4bL/jSPbz6ziGdvOYYZ8zeQmRLPOUdGzphLenI8f79pKt95fgkPzWl96TMzGD8onVtPHMaUob0orqjlk6Jy3ly2iecXljCybyrfP2s0Z47td9h0hxws5xxbd9WTmZLQ8nc0a2kZ33txCcOyUvnbTVMYkN5tvw/v16F0MXVb+7tIn9eGZaWSlhRH/vrtvLakjLIddTx141Run1HACwtLWgXErKVl7GlyHe72aPbkjVNoz3SHrNRE7jl/LAMzkvj5m6toaFzEI1dN5D9eWEzZjt08f+sxe80mHd4nlWdvmcaVj83n6sfn89RNUxnVr2u+bzjneOKjL1i1eSfHDs/i+BFZNDT6uO3pAnIyU3j06smkp8Tzf5eO587nFvP9l5byzsot3Hz80Ig7MyUpPpZHr5lE0bZqmnyOGDPMoG9aIhkpCa32vf7YITQ2+Xhn5RZ+/c7n3PZ0ARNzM7jjlBEcNyIr4v5tkaC2oZHvv7SUt5Zt9s9hSk9mYEYSC9dvZ+rQXjz+tTzSk7v3XIf9UUCEsL+L9HktJsY4OieDFwtKcA5+ecl4jh2RxUUTB/H8wpJWcyReXlTKmAE9Q/aZt0dHJ/l8/YRhJMbF8F+vreDUBz9gU1Ud93/1yJZ+/LZG9Utjxs3TuPYvC/jKwx9zx6kjuP2k4fts7Xy4ehv3zlrBbScO48qpB7f4inOO+95YyZOfrCc1MY5XFm0EIDk+loS4GP5yw5SW9++CCQMpLNnBEx9/AcDVB3lMr8XEWLvDNS42hnOPGsCZY/sxs6CUh+as4ean8umREMuJo/pw5rh+nHfUQJ1OC5Tt2M3Xn8rns807W34viytq2FBZy1VTc7n3/LGHfagqIEIo2lpNVmrCXt/Qusqk3Ew+WlPOjccN4fIp/vGBy/Ny+Pu8DfxjyUauP3YIa7bsorC0ip8GTk/tKtdNH0JiXCw/eqWQK6fkHPBDdezAnrzz3RP5n9dX8pt3V/Nm4SZ+cfFRTMrNaOn6aPI5Hpqzmt+/X0R8bAw/+cdycnulcOyIjp294/M57pm1nBnzi7npuKH89LwxfLZ5F58UlZO/oZKbjx+2V1fX3eccQXFlLT0S4xgSAd1gnSUuNoYrp+by1UmDmLu2gndXbmHOyi28vXwzH3y+jYeuOPqw7noq2LCd254uoH5PE3+5fgqnHNGxeUeHC+sul1TIy8tz+fn5nfJal/xxLnExxgu3Te+U1+uozVV1vFFYxg3HDmk1IHzewx/hHLz1HyfwwNuf8fhH65j/49Pok9b16+Burqqjb1pihybS/WvVFn7y6nI276xjYHoSx47I4phhvZlZUML8dZVcnpfN988azdWPL6C8up7XvnVcu+Z1gD8c/vPVZTy/sITbThrG3Wcf0aEPQOdct//A9PkcD7+3hofmrOFnFx3JdYfBTOBgNfWNvL18M68sKmXeugpye6XwxNfyGNlFXZ+RyswKnHN5oR7ztAVhZmcDvwNigSeccw+0efxE/JPtxgNXOudmBj12PfDTwN2fO+ee8rLWZi5whsh548M3WNk/PYmvn7D3RcCumJLDPa+toLB0B68uLuXkUX3CEg7gr7GjThvTj6lDe/HakjLmri1nzqotzCwoJTk+lgcvm9AylvLE1/K46NFPuPmpfF795rGBSWj7tr2mge+9tJT3PtvKHaeM4Htnjurwh313Dwfwd1XdeepIlpbs4Gevr2T8oHQmBGbvRyOfz2EW+v/dhooaXlm0kYqaerbX7KGypoElJTvYvaeJwb1T+I/TRnLDsUPC1ksQLTxrQZhZLLAaOAMoBRYCVznnVgbtMwToCXwfmNUcEGbWC8gH8vCfZlsATHbO7fNKdp3Vgiivrifv53O45ytjuen4oYf8ep2pqnYPU34xh2FZPfhs8y4evWYS50bxTGefz7Fq80569UjY6yyRuUXlXPfXTzl+RBa/vmzCPoNwUfF27nhmEeXVDfz0K2O47pjBh8WH/aHYUdvAeQ9/DMAb3z6+1Xrrka7J5/j0i0reKCzjn8s3Ex8bwzdOHs4VU3JIio+lvrGJP/97HX94v4iGJh+ZKQlkpsTTq0cCo/qlcfGkQUzKzdTvSJBwtSCmAkXOuXWBIp4HLgRaAsI5tz7wWNvLX54FvOucqww8/i5wNvCch/UC+5+UFm7pKfGcc2R/XltSRs+kOE6N8n7TmBgLeSE8gGNHZHHfheP46T+Wc/z/vcfleTnceuIwcnqlUFPfyNpt1Xy4ehsPzVlD//QkZn5jOuOzo/fbcFfKSEngj9dO4tI/zuO7Ly7hsevyIn7Qen15Dc8tLObVRRvZuque5PhYThvTl60767l31goe/aCIq6cO5rUlG1lXXsN54wfwX+eNPaiWrnzJy4AYBAQvPFAKTDuE5+518RMzuxW4FSA3t3POQPmi3L986LAwzIFojyvycnhtSRnnTxjY7c+wuGbaYI4Z1pvHP1zH8wuLefbTYvqmJbKpqq5lnzPG9uPXl07otks+emV8dgb3XjCWn7y6nKsfn8+j10wKy8I3/pUY6yiprKU0cJmR3XuaSEuKIzUxjhiDt5dvZu7aCmJjjFNG9+XCowdy2pi+pCTE4Zxj3roKHpqzht/OWc3g3ik8ddNUTgrDJNfuyMuACNWGa29/Vrue65x7DHgM/F1M7S9t33bu3gNArwhtdh8zrDf3fGVsVHctdcTwPqk8cMl4vnP6KP42dz1bdtYxvE8PRvRNZUTfNIb36aHugoN0zbTB9EyK54czCznvkY/54zWTyBsS+pTlztTkcywq3s47KzbzzsotbKiobfV4XIy1zA4HyM5M5vtnjuKyvJy9rpYcfNXi9eU19E9P6vZfnLqSlwFRCgRfwyEbKNvHvqGee3Kb537QKVUdQE1DE2aQFBeZv2QxMRZxYyNdoX96Enefc0S4y+h2zp8wkFH90rj16XyufGw+Pz53DDceO6TTL/PuC4TC60vLeHPZZsqr60mIjeHYEb256bihDOvTg+zMFAYEPuDrG5uoqW+itqGRgemhr1fVVnc6TTlSeBkQC4GRZjYU2AhcCVzdzufOBn5hZpmB+2cCP+78EvdWU99ISnzsYbcOghy+RvdPY9Ydx3PXC0v42Rsrmb18Mw9cclSnLXb093nr+dMHaymrqiMxLobTxvTlnCMHcPLoPvs8Qy0xLpbEuNiIbckfLjwLCOdco5ndgf/DPhb4q3NuhZndB+Q752aZ2RTgVSATON/M/sc5N845V2lmP8MfMgD3NQ9Ye622oezUmFMAAA26SURBVJGURM0flMNLenI8T1yfx0sFpfz8jZWc87uPuOuMUdx8/NBWc3FC8fkcby7bxMTcDLKD1mN3zvHbd1fz8HtFTBvaix+efQSnj+1Hqv6+ooYmyrVx53OLKSzdwQc/OKUTqhKJPlt21vHTfyzn3ZVbGNUvlR+fO4aTR/UJOdZTt6eJu15cwlvLNpMcH8t3zxjJTccNJTbGuP/NVTzx8RdckZfDLy4+SleXjVBhmygXjWobGklJ0Nsih69+PZN47LrJzF6xhQfeXsWNTy7k+BFZ3H3OEYwb2LMlKLbtqufrf8+nsHQH3zl9JMs3VvGLtz7j1cVlHNE/jVcXb+T66YO59/xx6rKNUvokbKOmvklNYDnsmRlnH9mfU4/oy4z5G3j4vTV85ZGPyUpNZPLgDMZnZ/DsgmIqaxr407WTOWtcf5xzzF6xmXtnreDVxTsP6pInEln0SdhGbUNjVM0sFfFSQlwMNx0/lEsmZfN6YRmLNmynoHg7s1dsoU9aIi/eNp2jsv2THf2hMoDjRmSxsmwnU4f2UjhEOQVEGzUNTWRn6m0RCZaeEs+1xwzm2sAF/sqr60lNjAs55yAtKZ5pw3p3dYniAX0StlFb30hKQmTOgRCJFFmp4blIpHStyL4ASxhU1zfSQ2MQIiIKiGDOOWobmtSCEBFBAdFKQ5OPRp9TC0JEBAVEK7X1TQBqQYiIoIBopaahEYAemignIqKACFbb4G9BqItJREQB0UpNvb8FkZKoLiYREQVEkJYWhLqYREQUEMGqm1sQGqQWEVFABKttHqTWGISIiAIiWE19cxeTWhAiIgqIIM0tCK0oJyKigGiluQWRHOIKlSIihxsFRJDahkaS42O1NKKICAqIVmoamjRALSISoIAIUlvfSA9NkhMRARQQrdQ0NJGiSXIiIoACopWa+kad4ioiEqCACFLT0KRTXEVEAhQQQWrVghARaaGACFKrMQgRkRYKiCA1DTqLSUSkmQIiSG29WhAiIs0UEAENjT4amnykqgUhIgIoIFrsDiwWpBaEiIifAiKgumUtCLUgRERAAdGitmU1ObUgRERAAdGipnk9arUgREQABUQLtSBERFpTQAS0tCAUECIigAKixZfLjaqLSUQEFBAtmpcbVQtCRMRPARFQq9NcRURa8TQgzOxsM/vczIrM7O4Qjyea2QuBxxeY2ZDA9gQze9LMlpnZUjM72cs6Aao1SC0i0opnAWFmscAfgHOAscBVZja2zW43A9udcyOA3wL/F9h+C4Bz7ijgDOBBM/M0zGobmkiKjyE2xrw8jIhI1PDyQ3cqUOScW+ecawCeBy5ss8+FwFOB2zOB08zM8AfKvwCcc1uBHUCeh7UGVpNT60FEpJmXATEIKAm6XxrYFnIf51wjUAX0BpYCF5pZnJkNBSYDOW0PYGa3mlm+meVv27btkIqtbWjSGUwiIkG8DIhQfTWunfv8FX+g5AMPAXOBxr12dO4x51yecy6vT58+h1SsWhAiIq15+YlYSutv/dlA2T72KTWzOCAdqHTOOeC7zTuZ2VxgjYe1BlaTUwtCRKSZly2IhcBIMxtqZgnAlcCsNvvMAq4P3L4UeM8558wsxcx6AJjZGUCjc26lh7UGVpNTC0JEpJlnn4jOuUYzuwOYDcQCf3XOrTCz+4B859ws4C/A02ZWBFTiDxGAvsBsM/MBG4HrvKqzWW19E/3Skrw+jIhI1PD0K7Nz7i3grTbb7gm6XQdcFuJ564HRXtbWVk1DowapRUSCaCZ1gAapRURaU0AE1Og0VxGRVhQQwJ4mHw2NPrUgRESCKCDwn+IK6DRXEZEgCgiCr+SqFoSISDMFBF+uBaEWhIjIlxQQBLUgNAYhItJCAUHQanLqYhIRaaGAwD8HArSanIhIMAUE/lnUoNXkRESCKSD48jRXtSBERL6kgODLLia1IEREvqSAQBPlRERCUUDgH4NIiIshPlZvh4hIM30i4l8LoodaDyIirSggCKwFofEHEZFWFBD4B6lTNUlORKQVBQT+QWqtBSEi0poCAq0mJyISigKCQAtCg9QiIq0oIPAPUutCfSIirSkg8J/mqhaEiEhrCgjUghARCeWwD4gmn6Nuj08tCBGRNg77gNBqciIioR32AdHQ6GNw7xSy0hLCXYqISEQ57L82905N5N8/OCXcZYiIRJzDvgUhIiKhKSBERCQkBYSIiISkgBARkZAUECIiEpICQkREQlJAiIhISAoIEREJyZxz4a6hU5jZNmADkA5UBTYH3w6+H2qfLKC8g4dt+/rt3edA2w50O3ibF3Xv6/HOqvtgat5fXQd6fF+/B6Huq+4D13Wgxw+m7s76m9xfXft7vD3bvKw7nJ8lGc65PiGP6JzrVj/AY6FuB98PtQ+QfyjH6sg+B9p2oNtttnV63ft6vLPqPpiaO7Pu/d1X3eGpu7P+Jg+27vZs87LuSPgsCfXTHbuYXt/H7eD7+9vnYI/VkX0OtO1Atw+l5vY8f1+Pd5e693dfde/7eO19/GDq7qy/yfY8/2D+JtvePyw+S7pNF9OhMrN851xeuOvoqGisOxprBtXd1VR3+HXHFsTBeizcBRykaKw7GmsG1d3VVHeYqQUhIiIhqQUhIiIhKSBERCQkBYSIiISkgDgAMzvBzP5kZk+Y2dxw19NeZhZjZveb2SNmdn2462kvMzvZzD4KvOcnh7uejjCzHmZWYGZfCXct7WVmYwLv9Uwz+0a462kvM7vIzB43s9fM7Mxw19MeZjbMzP5iZjPDXUt7deuAMLO/mtlWM1veZvvZZva5mRWZ2d37ew3n3EfOuduBN4CnvKw3qL5Drhu4EBgE7AFKvao1WCfV7YBqIInoqhvgR8CL3lS5t076/V4V+P2+HOiSUzM7qe5/OOduAW4ArvCw3ObaOqPmdc65m72ttJMdzEzFaPkBTgQmAcuDtsUCa4FhQAKwFBgLHIU/BIJ/+gY970WgZ7TUDdwN3BZ47swoqjsm8Lx+wDNRVPfpwJX4P7C+Ei11B55zATAXuDqa6g4870FgUpTV3CV/j53xE0c35pz70MyGtNk8FShyzq0DMLPngQudc/8LhOwaMLNcoMo5t9PDclt0Rt1mVgo0BO42eVftlzrr/Q7YDiR6UWdbnfR+nwL0wP8BsdvM3nLO+SK97sDrzAJmmdmbwLPeVdxyvM54vw14AHjbObfI24o7/Xc7anTrgNiHQUBJ0P1SYNoBnnMz8KRnFbVPR+t+BXjEzE4APvSysAPoUN1mdjFwFpAB/N7b0varQ3U7534CYGY3AOVeh8N+dPT9Phm4GH8Yv+VpZfvX0d/vb+NvtaWb2Qjn3J+8LG4fOvpe9wbuByaa2Y8DQRLRDseAsBDb9jtb0Dl3r0e1dESH6nbO1eIPtnDraN2v4A+3cOvw7wmAc+5vnV9Kh3T0/f4A+MCrYjqgo3U/DDzsXTnt0tGaK4DbvSun83XrQep9KAVygu5nA2VhqqUjVHfXUt1dKxrrjsaaO+RwDIiFwEgzG2pmCfgHFmeFuab2UN1dS3V3rWisOxpr7phwj5J7+QM8B2ziy1M9bw5sPxdYjf8MhJ+Eu07VrbpVd2TXHY01d8aPLtYnIiIhHY5dTCIi0g4KCBERCUkBISIiISkgREQkJAWEiIiEpIAQEZGQFBDSrZlZdRcf7wkzG9tJr9VkZkvMbLmZvW5mGQfYP8PMvtkZxxYBNA9Cujczq3bOpXbi68U55xo76/UOcKyW2s3sKWC1c+7+/ew/BHjDOXdkV9Qn3Z9aEHLYMbM+ZvaymS0M/BwX2D7VzOaa2eLAf0cHtt9gZi+Z2evAO+Zf9e4D86/C9pmZPRO4/DSB7XmB29XmX9VvqZnNN7N+ge3DA/cXmtl97WzlzMN/9VDMLNXM/mVmi8xsmZldGNjnAWB4oNXxq8C+Pwgcp9DM/qcT30Y5DCgg5HD0O+C3zrkpwCXAE4HtnwEnOucmAvcAvwh6znTgeufcqYH7E4Hv4F//YRhwXIjj9ADmO+cm4L/k+i1Bx/9d4PgHvLibmcUCp/HldX7qgK865yYBpwAPBgLqbmCtc+5o59wPzL8U50j86xYcDUw2sxMPdDyRZofj5b5FTgfGBr70A/Q0szQgHXjKzEbiv2xzfNBz3nXOVQbd/9Q5VwpgZkuAIcDHbY7TgH81MYAC4IzA7enARYHbzwK/3kedyUGvXQC8G9huwC8CH/Y+/C2LfiGef2bgZ3Hgfir+wAjn+iASRRQQcjiKAaY753YHbzSzR4D3nXNfDfTnfxD0cE2b16gPut1E6L+lPe7LQb597bM/u51zR5tZOv6g+Rb+NRCuAfoAk51ze8xsPf41vNsy4H+dc3/u4HFFAHUxyeHpHeCO5jtmdnTgZjqwMXD7Bg+PPx9/1xb4LxG9X865KuBO4PtmFo+/zq2BcDgFGBzYdReQFvTU2cBNZtY80D3IzPp20r9BDgMKCOnuUsysNOjnLvwftnmBgduVfLnK1y+B/zWzT/AvSO+V7wB3mdmnwACg6kBPcM4tBpbiD5Rn8Nefj7818Vlgnwrgk8Bpsb9yzr2DvwtrnpktA2bSOkBE9kunuYp0MTNLwd995MzsSuAq59yFB3qeSFfTGIRI15sM/D5w5tEO4KYw1yMSkloQIiISksYgREQkJAWEiIiEpIAQEZGQFBAiIhKSAkJEREJSQIiISEj/Dw0Py1Q7pJpYAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn.lr_find()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lr=1e-2/2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy_multi</th>\n",
       "      <th>fbeta_score</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0.088118</td>\n",
       "      <td>0.078493</td>\n",
       "      <td>0.960549</td>\n",
       "      <td>0.931567</td>\n",
       "      <td>02:17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.086540</td>\n",
       "      <td>0.078361</td>\n",
       "      <td>0.959779</td>\n",
       "      <td>0.932084</td>\n",
       "      <td>02:14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0.084194</td>\n",
       "      <td>0.075250</td>\n",
       "      <td>0.961298</td>\n",
       "      <td>0.935064</td>\n",
       "      <td>02:14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>0.082057</td>\n",
       "      <td>0.074505</td>\n",
       "      <td>0.961945</td>\n",
       "      <td>0.935554</td>\n",
       "      <td>02:13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>0.079987</td>\n",
       "      <td>0.074126</td>\n",
       "      <td>0.961218</td>\n",
       "      <td>0.935812</td>\n",
       "      <td>02:13</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn.fit_one_cycle(5, slice(lr))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "learn.save('stage-1-256-rn50')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "learn.unfreeze()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy_multi</th>\n",
       "      <th>fbeta_score</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0.086879</td>\n",
       "      <td>0.079334</td>\n",
       "      <td>0.961305</td>\n",
       "      <td>0.931122</td>\n",
       "      <td>02:52</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.084317</td>\n",
       "      <td>0.078403</td>\n",
       "      <td>0.961603</td>\n",
       "      <td>0.932634</td>\n",
       "      <td>02:52</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0.081326</td>\n",
       "      <td>0.077588</td>\n",
       "      <td>0.959685</td>\n",
       "      <td>0.934119</td>\n",
       "      <td>02:52</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>0.077480</td>\n",
       "      <td>0.075830</td>\n",
       "      <td>0.960738</td>\n",
       "      <td>0.934309</td>\n",
       "      <td>02:51</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>0.073812</td>\n",
       "      <td>0.075673</td>\n",
       "      <td>0.961283</td>\n",
       "      <td>0.934396</td>\n",
       "      <td>02:52</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn.fit_one_cycle(5, slice(1e-5, lr/5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD4CAYAAADlwTGnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO2dd5hU1dnAf2fK9l0Wlt7cRXrvoCiCvcQSK4iJMSYmGvN9sSQSEzUaNZqYZuJnolGjxtg1asSGiqACAiqwiPQFlrqUbWybnTnfH/fe2Tszd3Zml9k67+955uHOuefeOYeZPe89b1VaawRBEITkw9XWAxAEQRDaBhEAgiAISYoIAEEQhCRFBIAgCEKSIgJAEAQhSfG09QCaQvfu3XV+fn5bD0MQBKFDsWrVqgNa6x7h7R1KAOTn57Ny5cq2HoYgCEKHQim13aldVECCIAhJiggAQRCEJEUEgCAIQpLSoWwAgiAITcHn81FcXExNTU1bD6VVSEtLo3///ni93rj6iwAQBKHTUlxcTHZ2Nvn5+Sil2no4LYrWmoMHD1JcXExBQUFc14gKSBCETktNTQ15eXmdfvEHUEqRl5fXpN2OCABBEDo1ybD4WzR1rnEJAKXUmUqpDUqpzUqp+Q7nU5VSz5vnlyul8s32FKXUE0qptUqp1UqpWbZrJpntm5VSD6pk+paEEA5W1vLfNbvbehiCkHTEFABKKTfwEHAWMBKYq5QaGdbtauCw1now8EfgfrP9+wBa6zHAacDvlVLWZz4MXAMMMV9nHt1UhI7KLS+v5fp/f8GHX+9v66EIQsIpLS3l//7v/5p83dlnn01paWkLjKiBeHYAU4HNWuutWus64Dng/LA+5wNPmscvAaeYT/QjgfcBtNb7gVJgslKqD5CjtV6qjYo0TwEXHPVshA7J4o0lAFz1zxVtPBJBSDzRBIDf72/0ugULFpCbm9tSwwLiEwD9gJ2298Vmm2MfrXU9UAbkAauB85VSHqVUATAJGGD2L45xTyFJqPMHADhxSPc2HokgJJ758+ezZcsWxo8fz5QpU5g9ezaXX345Y8aMAeCCCy5g0qRJjBo1ikceeSR4XX5+PgcOHKCoqIgRI0bw/e9/n1GjRnH66adTXV2dkLHF4wbqpJsPryMZrc/jwAhgJbAd+BSoj/Oexo2VugZDVcTAgQPjGK7QkaisrQ8ej+7XpQ1HInR27nxjHV/tLk/oPUf2zeGOc0c12ue+++6jsLCQL7/8kkWLFnHOOedQWFgYdNV8/PHH6datG9XV1UyZMoWLLrqIvLy8kHts2rSJZ599lkcffZRLL72Ul19+mSuuuOKoxx/PDqAY46ndoj8QbrEL9lFKeYAuwCGtdb3W+gat9Xit9flALrDJ7N8/xj0B0Fo/orWerLWe3KNHRDI7oYNz+Ehd8PjhRVtYvbOUTfsqqKsPtOGoBKHlmDp1aoif/oMPPsi4ceOYPn06O3fuZNOmTRHXFBQUMH78eAAmTZpEUVFRQsYSzw5gBTDEVOHsAuYAl4f1eR24ElgKXAx8oLXWSqkMQGmtjyilTgPqtdZfASilKpRS04HlwLeBvyRkRkKH4vJ/LAt5f/5DnwBwbI9M3r9pVhuMSOisxHpSby0yMzODx4sWLWLhwoUsXbqUjIwMZs2a5ejHn5qaGjx2u90JUwHF3AGYOv3rgXeA9cALWut1Sqm7lFLnmd0eA/KUUpuBGwHLVbQn8LlSaj1wC/At262vBf4BbAa2AG8lYD5CB2PnIecf8paSIwQCmttfK+TrvYndtgtCa5KdnU1FRYXjubKyMrp27UpGRgZff/01y5Ytc+zXUsSVCkJrvQBYENZ2u+24BrjE4boiYFiUe64ERjdhrEKSUVJZy1NLt/POur0sv/XUth6OIDSLvLw8ZsyYwejRo0lPT6dXr17Bc2eeeSZ/+9vfGDt2LMOGDWP69OmtOjbJBSS0WypqDANxdV3j7nKC0N7597//7diemprKW285Kz8sPX/37t0pLCwMtt98880JG5ekghDaLYer6mJ3EgSh2YgAENotBysNASBZQgShZRABILQpHlf0xf3pZUUAeN0iAAShJRABILQps4b1YFTfHO75ZqQ/wCebDwLgkh2AILQIIgCENqXOr/G6XcybdgyDumc69mlslyAIQvMRASC0Kb76AClu42e49cCRkHMFpkBwiwpIEFoEEQBCm+LzB/B6Qhf4/l3T8boVBytrAfC45GcqJA9ZWVkA7N69m4svvtixz6xZs1i5cuVRf5b8ZQltis8fwGvuABbdPItBPTJ55brjyUz1UG7GAew6nJiwd0HoSPTt25eXXnqpRT9DBIDQplg2AID87pl8cNMsemankeZx2/oEWLe7rK2GKAhHxS233BJSD+BXv/oVd955J6eccgoTJ05kzJgxvPbaaxHXFRUVMXq04RxRXV3NnDlzGDt2LJdddlmrpoMWhITxxY7D3PjCat78nxPISPHg8zfYAOykp7hD3i/ZdIBRfSVdtHAUvDUf9q5N7D17j4Gz7mu0y5w5c/jJT37CddddB8ALL7zA22+/zQ033EBOTg4HDhxg+vTpnHfeeVFjXh5++GEyMjJYs2YNa9asYeLEiQkZvuwAhFbl56+sZduBI6zeaTzRGyqgyB99mjdUANz31tetMj5BSDQTJkxg//797N69m9WrV9O1a1f69OnDrbfeytixYzn11FPZtWsX+/bti3qPxYsXB/P/jx07lrFjxyZkbLIDEFqVLuleAGrqjfw+R2rr8TjsANK88mwiJJgYT+otycUXX8xLL73E3r17mTNnDs888wwlJSWsWrUKr9dLfn6+YxpoOy0RES9/ZUKrYun7a31+auv9HKiso2+XtIh+2WneiLaiMDdRQegozJkzh+eee46XXnqJiy++mLKyMnr27InX6+XDDz9k+/btjV4/c+ZMnnnmGQAKCwtZs2ZNQsYlAkBoVTymuqeqzk9plQ+ArLTIjWiv7NSItlkPLKK2XjKDCh2PUaNGUVFRQb9+/ejTpw/z5s1j5cqVTJ48mWeeeYbhw4c3ev21115LZWUlY8eO5be//S1Tp05NyLhEBdQJWFNcitftYkSfnLYeSkysHcCNL6wOtj372U6umXlsSL9wI7DFlv1HGNk3hxqfn837K6WOsNBhWLu2wQDdvXt3li5d6tivsrISMIrCW2mg09PTee655xI+JtkBdALO++snnPXnJQQCuq2HEhMnj58Zg/Mi2sKNwBbfemw5ZdU+ht/2Nt/4y8dsPyhqIUFoLiIAOhFnP7ikrYcQEyePnyuPy49oi2bvOnikjmv/tSr4flepBIkJQnMRAdCJ+Hqvc93R9oTbIa2D09P+3jLDI2Lu1AER57JSGzSXf3h3YwJHJ3RGtG7/O+NE0dS5igAQWpWXPy+OaHMSAJVmGohZw3pGnOtj8xpauf1wAkcndDbS0tI4ePBgUggBrTUHDx4kLS3Sqy4aYgTuZGwpqSQzxcNljyzlyaumkh8lxXJ7wsnn3xIKXTNSIs7Joi/ES//+/SkuLqakpKSth9IqpKWl0b9//7j7iwDo4IS7RZ7y+4+Cx3/7aAv3XZSYiMFE0S0zhVOG98Qf0LzyxS7AeQdw1/mjmDAwlyn5XblqRj5PfFIUPOeW+gBCnHi9XgoKCtp6GO0WEQAdnPkvR89t0h53vXX1AbLSPNxx7iiuPD6fnYergq6hdvKyUvneiYMAIs6vKZbEcIKQCMQG0MF51XyKduKLna2nKqmo8XHbfwqpqqt3PP/iyp089OFmKmvrSTUzfY4bkMs3xvaNee9oFcG6ZxnqoWTQ7wpCSyA7gE7Mxn2VrfZZjy7eytPLttOvazo/POnYiPM/fakhdD3QxAW7W2akHQAagsXW7ipjbP/cJt1TEATZAXQa/nTZ+Ii2aDV2WwTTcb+6zo8/oPnTwo3Bil7hjOrbtIjlK4/P5/ZvjAxpy0r1cONpQ42PRmwCgtAcRAB0Ei6Y0I8bTh0a0ja1oFtc12qt+XjTgaNSpaR6jJ9SnT/Akk0l/GnhJn7936+i9HWO8o2G1+3iuyeEGvLcLkX3LCNfkOQHEoTmIQKgg9MtM4Urpg8EYGBeesi551bsjOsez6/YyRWPLef11bubPQ5LADy8aAuLNhgud/4o8qS5qZ6H984OHpdV+4KCpLY+0Kz7CUKyIwKgA1PvD1Be7SPTjIxNa+KTtcVuM53C1pLYeXX2V9Q4qnZSPA0/pX9+WgRAbnpkSmdoug3A4u2fzGRor6zge0uQ1PhkByAIzUEEQAdmf0Ut9QFN/64ZQKQ/fU+HlMpOpHrjf5Kees/7TLp7YUS7U5K3vCxn4+3RcMX0Y4LH1g6gxic7AEFoDiIAOjA+v7HwZZgLeKon9OvcX1FLaVVdzPuUVBhP9LF06f4mZhutboEnc/sux9p1zH8lMcUxBCHZEAHQgfGZSnaryEqqQ0TtRQ9/GvM+lsom1g7giM3Hf94/loWoXnwOwqGmzjgfbly2dizNISPVmGNWqicYH1BRU99k4SQIggiADs2lfzcKSliRspYaZlivbMb2NwqlbIlDr2/hiyEArJ0CwCebD/Lg+5uC7+v9kddWmQJgv+26t/73RIb2yo7oGy85ZqnIob2yGNCtQZDslrTQgtBkRAB0YA4dMdQ7lgCwCq1npXm487xRAPTKMewAb6zeTUWNz/E+uRnGoupzWMS3lFQy6va32V1aHZJnCOBwVcP96h1cfiwV0JWPfwbAd47PP+qqZVbwV/iniSuoIDQdEQCdAEsFNK5/LhdO6McfLh3HhIFdOSYvg0nHdGXTvgp+/OwX/PRFZ135+AFGFG21z88Tn2xjziNLg9XFTvn9Rxyp83PPgvUR15XbBMrC9fsAY/dhYamIrDoF0VI6NAXrHjlhRePFFVQQmo4IgA6KvfxjqqUC8rj4w2XjOSbPiADefrCKBWv3UrjbSJ6283CV472sJ/9qX4A73/iKZVsPceBIqKunk4qnuq7hqXv5tkMAnDike8P5FjACjx+Qy82nD+V3lxhZTh+eNxEwkswJgtA0RAB0IP7zxS5ufP5LAEbd8U6wPTO18ZRONzxvFGCPZif11RsnamwL+rrd5fzspYbC7akeN13C/PrtAsDCnqo5/Hy0Mo9NQSnF9ScPoWe2UfQixxyTCABBaDpxCQCl1JlKqQ1Kqc1KqfkO51OVUs+b55crpfLNdq9S6kml1Fql1Hql1M9t1xSZ7V8qpVYmakKdmZ88/yWvfLGLytr6kKfrWALAYv2ecsf2WvPp/rOiQ8G2q55YwQsrG6p3+bWOyMNfFeMJv9r0z7fcNY/tkdVY92Zh3bsjlMMUhPZGTAGglHIDDwFnASOBuUqpkWHdrgYOa60HA38E7jfbLwFStdZjgEnADyzhYDJbaz1eaz35qGaRZGw/2ODZ07dLGgVNSPq2ouhQxNNyLO8fgDfX7OHQkTq+fdwx/J+pdhnQNZ38+W/y0Ieb6ZebzkUT+wftEdBgAzhtZC8ALpsSWd/3aLE8n+54fV3C7y0InZ14dgBTgc1a661a6zrgOeD8sD7nA0+axy8BpyilFIazRqZSygOkA3WA82OoEDfnPPhx8PgfV05pUoWsS/62lNv+UxjS5uT9E43eXdI4e0wfPC7FV7uNr/J372xgV2k1KR5XSPEWSwVU7w8wrFc2KhE6oDDsKSim3/u+eAMJQhOIRwD0A+xZxYrNNsc+Wut6oAzIwxAGR4A9wA7gAa21pWfQwLtKqVVKqWuaPYNOyKrth7nmqZWOOvZwMlKanv/n480HuPH5L/lk8wGgaQKgwDQwp3pcbD0QGmOQ6nGFLMhWcZi6+kBIeyLx2nYce8tr2Ly/9WogCEJHJ56/SqfHtnBzYrQ+UwE/0BcoAG5SSg0yz8/QWk/EUC39SCk10/HDlbpGKbVSKbUyWQo7//G9jbz71T62HYgdxNWc+rjVPj+vfLGLef9YDjTNgGoZgp2ijlM8LmYc2z14XF5Tz89fWUudv+UEwKDuoXaFqjiEpiAIBvH8VRYDduVtfyA8b3Cwj6nu6QIcAi4H3tZa+7TW+4FPgMkAWuvd5r/7gVcxhEUEWutHtNaTtdaTe/ToEe+8OjSHzfw99YHGF+ZThvekf9f0qOevnRVZmQsaAsgs6qLlbQ7j0sn9mTYoD3DOC5TidjFuQC5b7j07KFSe/WyHsQNwSBaXCFwuxdT8hroHNT4/lbX1/OG9jQy/7a0m7W4EIdmI569yBTBEKVWglEoB5gCvh/V5HbjSPL4Y+EAbCWB2ACcrg0xgOvC1UipTKZUNYLafDhQiAA3BTvbgJid10M/OHN6oXv3m04eRnxc7747PH+CU4T1j9vvtxeOCOw6nXYP1lO92qZB4gJZUAYHhoWRRVefn2n+t4sH3N1HjC3D4SOxkeIKQrMT8qzR1+tcD7wDrgRe01uuUUncppc4zuz0G5CmlNgM3Apar6ENAFsbivgJ4Qmu9BugFfKyUWg18BryptX47gfPq0FiLrD3Z2lX//MyhX+z7xHK91FpTVu0joxFX0nSvm5evPT6kzSnIy77I//Vyw1NoeO9sVheXtagAWLX9cPD4SG09SzYdCL4vj5L+ws7WksqoxewFoTMTlwO51noBsCCs7XbbcQ2Gy2f4dZVR2rcC45o62GTB4zIWy1pbnvtlWw9F9HO7Yi+qsWwEH2007CpvmNXAzh/fl9e+DNXwVfv8TDqmq+P1Jww2nvQ/3nwgxAOoS7qXcQNyWb2zFKBFBcDPzhzGb9/eABgCwE5lbeM2gcNH6jj59x9x3KA8nr1meouNURDaIxIJ3A7ZXWZktqxxcGm8cGKDA5Y7DrfKWALg2n99DsAlk/qz+Z6zHIvLN8ad549iQDfDDpHiDv2sdFvpx9QWsgEAXDdrcPD4SJiqLFwghDPh1+8BsHTrwcQPLAYffr2fd9btbfXPFQQLEQDtkOLDpgBwqHRlpUAAcLvjFwBnj+nteN5S5QzrnY3H7Wqyr36P7FRc5jXhZuF0m6dQS1QHcyJ8wf/uP1e0yuc2h6v+uYIfPL2qrYchJDEiANoxTgbMvrkNAiC8ApgTlkH5tJG9KLrvnKj95k07Juo5JwaaufjTPO6gkAn3DFpR1KCbb61UDXvKakLex5slNC+zdQSUILQnRAC0Q3qYtXwfWrSZfeU1wbaMFDffnNCgAuqeFbvmr+W374ryZN81w0u/3PRgnn07188e7HCFwXPXTOfPc8aT4nFFFQCVtqfxsurYxtij4bszCgB4aVVxxDl7RbJoMQ8HW8lb6GBlLQcra2N3FIRWQARAO8RapEqrfNzyspHDv7rOz5wpA8lIiS/xm8XNZwzjxycP5uwxfRzPawz1jxNWjMHsYZHxF31z0zl/vCGMLB//ukZ87rPiTFjXXG4/dySDejjnRDpQaSzun24+wNBfvsXnOw7j8wf49/IdIf2Wbml5O8Ckuxcy6e6Fkr1UaBeIAGiH2PPZrNtdjtaaI3X1ZKW6mxz5m53m5abTh4V46IBRnQsMIRO+O/jlOSMAY9ex9d6zefw7Uxr9jN5dDLWUvWA7wAXj+waP0x0ihxNNtGCzKfcsZFdpNV+YHkkX/t+nnP7Hxdz66tqQfh9tLGHHwaq4IrCPlvBgPEFoC0QAtDO01iHG35KKWqp9frSmUV/9pjK1oCF6NvzJ/bszCvj396Zx8vCeuFwqpmH4W9OP4a7zR3HF9FA7wuh+XYLH9100NgGjbpxwO4PddXXGfR+EnLMv8j1NldvfPtrCzN99yOwHFrXcIE0OiBpIaAeIAGhn/HHhpoi2x5ZsA2DRhv3BNquMY1PJTjOEiH1HcNNpQ0P6uFyK4wd3j9sjyON28e3j8iN8/a2dxXeOzw/aNVqTn5w6JOT9797Z4Nhv+FHWKW4OjyzeGjx+eVUxOw85V2sThJakZRWzQpN58H1DAFwxfSD/WmboqNfsMko69soxVC3r7zqzWUngAN674SSKD1dRYRpoUz1G/p6WwJIfdiNsaxJeNzgaM47NY/HG1k00+PrqhmC7m15cTYrbxcZ7zorop7XmjD8tZtIx3fjNhWNac4hCEiA7gHaKxxbl+95XRsH1759oJFJNT3E3O7K2d5c0Jud3w2tFG7egMdISUW2x/N9x7shguchYWP+vduxFdxJBZW09+fPfjHo+mgH91lcL2bivkmc/2+F4XhCOBhEA7Qj7k7LWmvduCM2QncjFujVy36SYRuGWygQajW2/OZurZhSQkxbfBtflsJt64pOihI4pnqR0Tgn/ZOEXWhIRAO0I+wLv1zrC6GsFXyWClqjOFc7Fk/pz7axj+UmYjaGlseYW7w7AiUSnkXZS2fXLDU3l/fa6PQn9TEGIhQiAVmDd7jIm3PUuu0uNFA9/eX8Tf/9oS0S/RRsa9ND+QGik77gBuQk1pAZaQS+f4nFxy5nDWzwGwCJcpoW7vjZGeO2ERMtHp/oJXcIE1GfbIhP+TTO9tVrDjVZIPkQAtAJPflrE4Sof739tePH8/r2N/OatryP62aNYR/XNCdHz5x7F06wT1vp/5ijnHEEdkSU/m80r14WmrW6sYM6kY7ryxFVGjMP/nhLqMaQci9w1Hye///BdwbOf7YzoY/0GMlPdXP/vz8VbSEgoIgBaAeuPOJY3jBUAlpeZwrxpA0N051+aQUyJ4uThPZkzZQB3nj8qofdtS/p3zWDiwNC01e/dcFLI+wkDGzyebjp9KLOHGYVw0rxuvrjtNO6+YDTQENyWKKyIbjtOtodwrJTgByrr+O+aPdz2mtRNEhKHuIG2ApY/fMBBDWBnan43lmw6wH9+NAOlVIgKKNa1TSXF42qV4Ky2Jjxh3ss/PJ6Sylo+3XKA48zylhZdM1O4fOpAfvmfwoSnarByOtnxOAgArXWIfSa88I7UPBYSiewAWgFLAPg1bLBFq36y+UBIvyqfH69bBdUW9oUg1StfVXOwP2UX3XcOLpeiV04a35zQ39EQ7nIpvG7VaF6j5nDm6IZcTI98axJgCIA5UwaQaUvEZ48C11qz1owBsXCyEwhCc5FVpRWw1pnaej/Fhxt0uC+sDNX5llX7yEnzOi5M4wc4V+QSYjN36gB+de7IuPunuF0J3wHU2wSK19yVWLuwT39+SvCc/Yl/h+j7hRZGVECtSE2dn1RbwrRwz5Dyal+EZ4iFRIE2n99c2DRVV4qnBQSA+V2/8IPjyM0wvmMrQ6vdw6eqrp5uZm2CaLEIK4sOMTm/m+M5QWgKsgNoBazFpKY+QJ2/4QlvbP8uIf3Ka+rJjiIActJFVrcWqR53SEbWo+VIbT1f7S5nUI9MphZ0Y2ivbFb+8lTmTh0IhNZLtgeDTTEX+eywgLbFm0JVh4LQXEQAJJg3Vu/mG39ZErLlf8bMO19d5+eIrUj59oMNW/wan5/FG0uCsQLhtHY0bTKTm+HlcFXTCths3FfBfgdDL8AVjy1nw74KMm21HKIV8wk3+kJkYZ4H39/EnxZuZOO+1qmyJnReZFVJMD9+9gsKd5Xz42e/iDi3p6w6JAXDM8t3BGvYfrWnHDDSPzvRGpG7gkH3rNQmp2s+/Y+LmXrv+8H3dpffL3YYLryNuZb++3vTgFAvn/qA8RDhFNH8p4WbmPPIsiaNURDCEQHQQtgNeFa++YXr91NeHZqD56BZrep3bxupiqfki7G3remeldLsfP0HKmv56webGH7b2xE2nsYiudNMTyD7DqDeb1wfLZJaisoIR4sollsIu495hs3N754F60P6VdT6+OsHm1i61ShH+MOTQlMSCK2PUoqdh6qpqPGRHUdKaXtxmcl3Lwwe76+ooU+XhkjkcF2+Hes3YrcB1Jh2iKxGrquu8zvWcxaEeJAdQIKxnuDtid0ay+L5wDsbeODdjcH3zU3zLCSOV7/YBcBTS7fH1T+8gliaGbOxu7Q6qOIDSG3EjmN5AtkFwC9eLQw558TZDy6Ja4yC4ISsNgnG8vj5YufhYNuR2np654Tqf7uaroAfbggtRCLG3rbnD5eOA2B5HEFXNQ5GWyuYq7ouwF1vfBVsbyw5nfUUX2W73wizUtnEgV356RnDWH7rKZwxqlfIda1Rv1jovMhqk2AsI17hrnL8AY3Wmqo6P+MGhLp8Pn31tLju99b/nhiR4ExoWSz//MUbS2IWhqmsjV5XwecPsPVAZfB9ZiNZURt2AA3321JiXJvicfGj2YPplZPGA5eMiz0BQYgTEQAJxu7FUVHjo7Y+QH1Ac2yPrJB+GVH0tuEpf0b0yYlIcCa0LGk2lUs0rywLpzTPFnX+QNDIDzBv+sCofTNMF9HVOxtSPzgFozkZhPeUObsOC0IsRAAkGLsXR1m1L6gDtvt933r28OAffDhtVT9XcKY+RhK+xgSAzx+gr1n05e/fmhQSBR6O28xBZNVpiPY7cHIHjhY7IgixEAGQYOxGvNIqH5v3G9t4ezK3q08YRGaq82IwJiw6WGhbnAKz7Nz5xrrgcXbY0/na4rLg4nzy8J4xP2v8gFxKzQC0FUWHY/RuYO4jy+PuKwh2RAAkkEBAU+3zc8Lg7gCUVvuCC8iQntl874QCeuek4XapkKhQi6L7zonL7VBoPZzq9Np5Z90+wCgu88Xtp4WcO1BZx8i+hiE3nupkXdJTKK02BMCRRmwLd5w7kqxUT7B2QaIzlwrJgwiABGL5bVsRn6VVdcEFJDPVzS+/MZJltxqZH10uxXW2MoSJrPcrHD1/mTsBiD//fk6aB49tkR/cM4tqXz0pHldE7d9o5GZ4Kauqo6SiluJG1DpXzSig8M4zou4iBSFeRAAkEGux6GMKgLJqH9c+8zmAo/73Z2cODx6/cf0JrTBCIV6OP9YI5Curji8nULiHT0aKm+o6P4GAdiwI70RuupfSah+zH1jEbf8xYgC6Z6VE7X/euH5A9EhhQYhF0giAdbvL8CV4q1xXH+BXr6/joJk2oDooAIwnvsc/3hbsG+1pbaJZorBLhqh+2hPdMlPITvPEdAO1Fvf8vEwAPr5lNu/fdBLpXjdVdX782rnylxO5GV6q6vwhrqWPf2dKo5/93RkFcd1bEJxICgFQdOAI5zz4MfeGpWE4Wv67Zjf//LSISXcv5B9LtlJeYwxKfo8AACAASURBVDwtWqmbi2zZPntlOycCe+rqaXx486yEjks4epRS5GWmRM23c7Cylrr6QNC4++NTjIyd/btmcGyPLNJT3FT7/PgDgbhq/wJ0yYh82s+LkjXUoiVqFwjJQ1LsHQ+af8SJLqxeUdPwpHb3mw3CxcnHP9oikJXqkS18OyXN6w4p0WihtWbS3Qs5dURP6gOasf27RKj4MlLc7Drsxx/QuOPM5JrrkPUzM0aenxSzfKXkBBKaQ1LsAFrKtz7afdO9oQv64J5Zjv2E9k2qx+XoYWN5di1cv5+6+oBj+o40SwUU0HHvAHId1IDR4kUsrNxRE379blyf0Rg1Pj+n/uGjiFrVQuclKQTA08uMpF5bSxKbNyVaDFD4DqCr6Pc7JIZ6xVjs7QFf9mIxdfWBkBgPi4wUNzU+PwvX72e9WeshFrnpkSqgWMkBg9XmHHYqTaX4cBWb91dy+2uFR30voWMQlwBQSp2plNqglNqslJrvcD5VKfW8eX65UirfbPcqpZ5USq1VSq1XSv083nsmkiVmCb2yal/QELxxX0WjUZzx4I3yxxlewCPXQbcrtH9SPC4qa+sp+PmbHHvrAgp3GWkaDtvsAmXVPtIcPLzSvW4qGvHld8JpBxCLzSWVsTvFidtl/J5jRT8LnYeYAkAp5QYeAs4CRgJzlVIjw7pdDRzWWg8G/gjcb7ZfAqRqrccAk4AfKKXy47xnwrB7Ydzy0ho276/g9D8u5s8LN0b0PVhZG7e3ULSNfe+cNAq6Zwbfyw6gY5KbnkLhrnIsTd9HG43MrfYgrU37KxndLzJ6Oz3F02TjbGP1AqJhj1M4WlWn36xAZhWiETo/8ewApgKbtdZbtdZ1wHPA+WF9zgeeNI9fAk5RRtISDWQqpTxAOlAHlMd5z4RhFwCvr97NnjKjdusXYUZhy7j3I9N3Pxb2PP8nDjGif3tmp5Ke4uaamYOC575zvLjqdUQ+KwpNB21l7AwPDrOifZ36NoUuDkbgWFhR5wDFh48uJ5ClRjranbHQcYhHAPQDdtreF5ttjn201vVAGZCHIQyOAHuAHcADWutDcd4TAKXUNUqplUqplSUlJU5dYtLLVou1PqCDuvvwxFqfbDaqcr371b647vvr/xq53j/66axgxs79ZvbIuVMbMj86LRBC+yc8gtfy1jpSF6racVrs0x3sArFQSnHqiF6xO9q4+oQCfnrGMCB23qJYWAZvqxax0PmJ51fqpOkIf0SI1mcq4Af6AgXATUqpQXHe02jU+hGt9WSt9eQePXrEMdxIxg/IDXl/5eOfAUa+9437KoLtG2zHTaF/14yEB5kJbU+4SubtdXuByB1AmoMAiOW9E41Hvz2J926YGXd/pVTwAaOx2gTxYKmsZAeQPMQjAIqBAbb3/YHd0fqY6p4uwCHgcuBtrbVPa70f+ASYHOc9E8YTnxRFPXfLy2uCx4NsevtFG/Y3mgagyvYU6HYprM1ET1vh73d+MpNXpZhLh+Unpw4Jef/B1/sB+I9ZMtIivNobEJLLpynRukophvTK5q7zR/FEI1HAdqydSWVN/AKgcFcZD76/KaTNUmk6pZxuTQIBzdd7y9lXXtOm40gG4hEAK4AhSqkCpVQKMAd4PazP68CV5vHFwAfasEjtAE5WBpnAdODrOO/ZKthzqdfWNzzZfeeJFdz4/JdRr7PS9lp62/PH96N3ThpPfndqsM+w3tlMkGIuHZZJx3QLHp83rm+wpoMnzO+/Z05ktK7dU+j2c5vu3/Dt4/KZHUcKaSCYWbaxDKLhfOMvH/OH9zay37bIWjuAtl3+YdCtCzjzT0uYdu/7bTySzk9MAWDq9K8H3gHWAy9ordcppe5SSp1ndnsMyFNKbQZuBCy3zoeALKAQY9F/Qmu9Jto9EzivuDlzVO/gcXjx9k37o7vYWWkffnPhGACG9spm2a2nBOu4tilSVCbhVNXVc6CylsraejburQiJ3nZSAVmpIVoDS1XVHBWQvfC99QB08EgdAVEDJQVxKSq11guABWFtt9uOazBcPsOvq3Rqj3bPtsBn+6GH5363u29u3l/JjkNHOGFwD373ztdMyTeeDnPaY/7+lY/Dh/dAt0HOr4xuse8hhLBwvaH+Of+vH7M3DtVEz+w0/nTZeMa2QoEfKxNpcwSAvU5BVW3D7/9wVV3MPEQtQbgr64cb9vPeV/u454LRba6a6owkfRIan+2pP9yLIsXjYuehKn7w9Cq+MqM5/3r5BB5dso1HlxiZPpvju93idB8CI86FQ1th+6ew5gVCbOxpudGFQ2Z3kD+0qGxpQjT5BRMcHdsSjpVpNl4VkN3IW1HTYOey27U27K3g+MGtLwDsQWj9ctP56YtrOFBZy49mD467roIQP+1w9Wp5umZ4g+H8du+dcAHgcbl4/JNtwcUfoLw69I+sXQqAgpnGy8JXA6XbDYFgf+1aCeteAW1TfaVkQ7cCZ+GQ3TtphcP/nDyYBz/YHNFu98NvK1I9brxuRaX5BL+vvIZp977PU9+dysyhkZ5zdjdWewK5I7Yd8Krthzm+DeZmV8NW1dUH/05LKmpFALQA7XD1annsP/T/fLmbP80xqj/VhKmA6gOBCA+iu/4baqporrtfq+JNgx7DjFc49XVQtjNSOOwrhK//CwGbwPOkm8LAQUDk9ANX500tdd3sUAEwul8Or1w7I+5iLy1NVqonuANYttWIZ/n2459RdN85If2WbjlIaVWDgdry/X/s42387p0NwfbmpKVIBLU+uxqqYXdi36kIiaMDrF5Hzzlj+/Dmmj3B93X1Ab5zfD7//LQopF/4DqBwV2QSr/CkW7GSdbV7PCmQd6zxCsdfD+XFNsGwzfj34GbY9B74axv6ulOha75NKNiERJcB4O6YP7UPb55FSUVthKH3pKE92tV3n5nqCdoArNTUTk/Mcx9dFvK+1vw9W0GNFmuKy1pimDEJd8SwiLc0p9A0OuZfZRNxStblcSl6ZqcGI3cn3/0eBypDi3/EE1nZnhaBhOP2GIt613w49uTQc4EAVOyGg1vCdg/bYOsiqLelJXB5IPcYZ7VS7kBDCLVTCrpnhuR1svjR7Nbz8omHLJsAKKs2fsfx1JmOtuC+uKqY310yLnEDjJNo45FAy5YhKQSAdggy9rhdnDy8Jx9uMLw7whf/aJw3ri+vr26IWUvtzAKgMVwu6NLfeA06KfSc1lCxN1KtdGgr7FgGdbaIa+UydghOwqFrvqG+amd0Sfe2O9Vfpk0FdN9bXwPEVSDGHvsSTl19oNUfcKzxfGv6McE07tZYhMTTvn7FLYWDS7PXrUj1uKI+cUQj/Ekk3nqvSYVSkNPHeOXPCD2nNRw54CwcCl+CGrvqQRm2BUejdAGkRD6ZtySW2tDrUACmrclK9XDY1O1buvN4fpqN/f73ltUwMK9hF1FVV9/igs9SSYXbILYdSGwtD8EgKQSAU0iLx+UixeMK/uDi5a3CvSHvxTe5iSgFWT2M18BpkeerDjXYGuyvr9+EqrBKVVm9w2wOBYaqKXcgZPZIuMeSZVw9UFkbo2frk5XqYedhowZ196xUDlTWxvVw8+aaPTx0eWjb/ReN4ZaX1zL794vYcu/ZALzyeTE3vrCahTee1KIV7qzAtPD4mr98sJmbTndwYhCOiuQQAA6RsR63ItXjdiz5J7QhGd2MV/9JkedqysKEg3m8eSFUhgpmPGmGail3gCEQugwwhYP5Pqt3k72Wrps9mP959oujmFzLkZnqDqqAumV6OVBZGyyE1PR7GcuCPV7gHTMR3qZ9FS0qAF7+vBiItL+dHGdaDKFpJIcAcGgrr/GRleLBH9AhATRzpgzguRU7g/864XUrfFI0o/VJ6wJ9xxuvcGor4XCR4dJautOIeyjbCaU7YM+ayN2Dy2vYL4ICYqDxr/U+u2+E59KEsKyy7Yl95bXsK6+lrMpHRZxJ4eZOHch7ZurzrFQPAa35/SXjSLPZDhZvLGHm0B7BFOrXPvN5hGtpS3DS0B58vOlAsCbDB1/v58udpRGZfYWjIzkEgMNa/fePtvLzs4YDUGrL+jlv2jH85sIxvPbl7qgCwOfXHH9sHp9uOdgi4xWaQWoW9B5tvJyoOwJlxYZAsF6WgNjksINQbsP+YNtBdM/sx/GufRTrHkb8RDvyXrKqlb3z1d6gZxsYmTWtovT2J/qhvbLITHEHo3/rAwGuPC6fs8b0CUmRXmLeq7yRzLiJwm6QHjcglxtPH8qcRxrcVu96Yx2vXDfD6VKhmSSFAIiG5cFj/bhvOXM4Y8zcLZah76zRvSP0/gD/unoaAUm61nFIyYweDAdGtHT5LmPnULozVEhsWwzlu0lH829rzb/7RsjuE6ZisnYQxxi7C2/rRa7mpHkor6ln1+Fq/AHNsF7ZbNhXwSNLtvLDk4wYD0utcsmk/tx+7kgeXbKNqjo/Ty8tosYXCAoKe6I7pQwhsnzboYjPTDTDfvl2yPvMMIPzkVo/+fPfBGiVXUgykBQCwL5Mj+6XQ+Gucn5z4Zhg2ltLANhz+Vvub+leN6eO6EWP7FQWrN0TrBHgcilcbZ44V0gY3rToAXFgPPGX72LuAy8yMr2U207IbhAQO5dD4Sugw1wqM3uGqpXsdoguA4xdS4J44qopXPTw0mA1rz65aWzYV8F9b30dFABWiuoRfXLITvOSaap6bnvNiG63MoBm2FRAKR5XhNF7a0klg3ok1g5gzz46d6pRKiQvK3SH1dyCTUJ0kkMA2J7UM7zGlLtnpVJZayzm1qJuj/Y8YXB3zhvXl+/PHBRM8fzx5pJGi8QInRhPCnQr4NE7bjTcK8PdIf31ULGnQa1kt0PsWW14MfnDYk3Su4XtGgaECoy0+DOJ9jWjfh/6cAvgnKX2zjeMaN831uzmuycU0DUjdIHdZdbGyM1IYe7UATz72U78AR1MfW7vl2gBsHL74eDxV7uNCPy+uek8/p3JXPPUqpAkcULiSA4BYDu2DFwBrYMh8+U1Vgh9g1dIeoqbB+dOCLnP9bMHc8vLa7mjGQU+hM6BXT0SgttjLtwD4BiHKnCBAFTuswkIm4qpZIORWqM+LM10aheHHYRNYKR3Dbq6hi/4OekN4/T5A3jdLhauNwy+F4w3spR2zw4VAP9ds4e/mi6h1588hGc/20mNzx/x0NMSa/Glf18aPLa7r548vBfH9siSp/8WIikEgF0CjOyTw+KNJXTLTAmqfqwfeGqMQt6XTRnIpZMHiO+/0HRcrobguAFTI89bAXKlO6Bsh20XscPwbtq2GOrCChR5MyFvEOTPJGPQSWRQQxVG5PTUgjz+tWwHAEN+8VaIznzu1IFAZAzL1PyGOhFZ5g5nw95KemSHpoV+8tMiTnLIMvrW2j0UHazi2llR1GhxUhPmAmoXZkJiSYr/WXsqiJtOH8qMwXlMye/Gx6afdLmDCigasvgLLYI9QM4pBkJrqD4c6r1UuhP2r4MV/0Ate4jVqW4+10P42D+akfVpuPHjx/hN1/sDjOnXhew0T9C+Ff5Ltpeu7JLhNd2dA8E0KRMG5vLFjlI++Ho/m/dXMLhndsj11z7zufHvUQqA8BiAedOOYUXR4ZC2ytr66LsxIW6S4n/Q7qzjdbs4cYjx9GI98VvFp5M2r4/Q/lGqIUguPA7CVw07lvGPJx5jhquQGzwv43rjJdZnZvJR3XA+Dozm8PYBrN1VGlLnODyz7eh+oTaHntlpVNX5+dlLawC495tjOOvPSwB4fsVOfnFOy6hCw8d1wYR+PPvZjhBPpKo6EQCJICn+B6N5a1rFXCx//1SHrKGC0O7xpsOxs7m/3kgFkUsF71ygqFy/kCHbPuI07yp46kmWpXZln286rN4BBSeFLPi3fSNyMU/1uFhR1LDo9s5pSMz36JJt/OCkY+neAmUjf3xyZKbV8NxA4eVbheaRHALAMRY4Ml1uWgwbgCC0Z0b2yeGrPeWUko1/+Ml8nTGTH234nP5qPxfnbubYypWcUbkMXn0LgP49hrNt5kmsTZvA6EmRdaK3hiVgy0kPXYRbIkPn01dPDe7Q7aSHqWfjSdUuxCYpVjxrB/DotyeHtIfXCQj/kQlCR6JrZsMCneJxBf35i3VP/nT4eH7s+x+WXrQcfrAETvs15PRFff4UYxf/ENdvC+Cx0+HDe4060vWR6dHDq59FK0L/8qriRsf5duFe8ue/yf7yGipqfGitOcXM9ROtxGa3TGOnkWPu2qVATGJIkh0ADO+dzWkje4W0u8J+0OHeDoLQkfjF2SM5+0FDR5/qcTk6NaR5PdBnrPGa8T9GBHTxZ0YRn62LYPHv4KP7wZvJ494hfBIYzceB0WzQAyLu9b0nV7L4Z7OB0N3ATS+u5qJJ/aOO84f/WgXA1HvfB+CX54zA63YxtFdWVCeLXjnG36ZlwA4v3yo0j+QQADq6906fLmnsKTOMwOLhI3RkRvbNCR7bdwB2IorEeNOgYKbxOuV2w9Oo6GPY+hFTvlrIyUf+BUCltxu8/B8WnTqFuQvT2UMeOw5Vsbesht5d0iIKy3y86QDbDx1h3rRjYo57wdo9HDpSx6Ej0Ysy9TLtD1aMgOwAEkNSCADQUZM2XDJ5AA++v6lVRyMILYVVCyDF7XKsCBZTzZneFUacCyPO5YN+u7j/uYXMcBfyP/13kbV1EflHXmRpGmwJ9OGTwGg+/E8Rcy+dR60/NO/RFY8tB2DulIERO+2ThvYIJq8DY4dedLCq0WH1NHcAVkEmsQEkhqSxAUR7uD9uUB4As4dFGp4EoaPx2vUzeHjeRJRS9OmSFpLfCuKLdbE4dUQvdtOdF/2z6H3Vv+DmTXDtp/w9/Xts1724yL2Yudtuhd8WkPP0GdzseZ7jXOtIoSFy+IT7P4i4b9cwj56yqtjpVXpmGzsAS9UU7gV0yd8+JX/+mxFBZELjJMUOQNOIADg2j9V3nE6X9MjcKYLQ0eiXm04/My9QdpqXpT8/hedW7OAXrxYCkZ48jZGZ6mFc/y5MPzavoQxmr1H84Jbfkz//TbzU88txR7igyyZqN37AD91vcL3nNap1CisCw/g4MJpPysdAYHZI8Z3K2tBF2vI2Gt47NLDMjrUDGNOvC6uLy3hz7R4unWLYJf67ZncwUGxNcRlTCyI9mgRnkkMAaI1qJHOnLP5CZ8XtUlw2eUBQADQ1eOq160+Ies6Hh5Juk5i8OBeffzJZVDHNtZ4TXIXMcBVyq/dZ4Fn0A79HFcyEQbMo6XFcMCdRONMaWbhz0ry8+MPjGNgtg2n3vs9HG0uo9wdQSnH9vxuqtF3696VsvucsPO2wbnN7JDkEAAkvDysIHQb7Yhjuynm0pHpcwep4lWSQMvIc7iycRL/cdHylu5nhKuQ3Iw+StmMJrHuVHsBHKT2D3kWfBkZRivHk7xSMZmdKfreQtNE19YFgQRs7VT4/OSIA4iI5BICOzHsiCMLRE27gtXbTe8qqCdCVVwMnMm/qcQw4I51edTv4zV8fZrJew7nupVzu+YCAVhTqfD4JjMZTlAEDpzdaSMf+edV1fk7/4+KIPpU19Y7psIVIkkMAgGwBBKEF2BSWptmyMQS0Yew9XOXj4r8ZqZ6L7juH/6R+g7+Xn0bvLA99j3zFDFchJ7gLudq9AJ5+A9yphhAYNMt49RkHLmfDdY3PT6mDAXn+K2u5ZFJ/zh3XN4Ez7ZwkzT5Jln8hmZk+qBuj++XE7hgnz18zHYBtB6tC4g3s9rRumZE1ky0bxNiBeXyuh/IX/4VcVnc742sfhXkvwdTvQ9VBeP9OeHQ2/HYQPP8tWPEYHNwCWnPlcUZswadbDjiObfHGEn787BeO54RQkmMHILV7hSTnuWuOS+j9pg3KY+7UAby5Zk9IUJaVqgEgLyuVLSUN+YSG/fItzhnThy0lR7jtGyN596sGY3AVaTDkNOMFULnfqIGw9UPYsgjWv260dxnIdzNGMtJTg+81N3d4PNThwYeHOu01/sWNDw+s3A/uFPPljXHsBU9q5HmXp1NrD5JCAECn/g4FoU3omZ0WrKZnYXcz7R5W07e2PsCXO0vJSfMwICwRYwRZPWHMxcZLazi01RAGWxfRc+dqTnRX4qWeFOrx4sdLPV4VFgPw3yePan5BYgkOd4qhuopHyDid96TEuM487jU6qjqsuSSFABAjsCAkHqdUE/27NizsJRW1EefDM4zGhVKQd6zxmvI9vP4Ax//irYhur147ncseXkIKPrzU8/mtJ6H8PvD7jHrM/roYx7WN9IlyXX2d7bgWaitif16gmXXFf7FPBEBz0GjJ8yMICcYuAO6/aAzfGNs3mFcL4q8d/NiVk5kwsGvcnxvNx9/lclOHlzqMXUh1Wk8yUtrhEqd1nAIp7NgdaVM5Wtrh/07ikR2AICSedNvi6nW7yEz1ELDZ2+xG4Hu/OYZbX13reJ9TRvRybG8q4QGdpVW+9ikAlDLsDZ62zz6cFF5AjeUCEgShedh3ANZTeUH3zGCb/fhih/TQdoPx0ZCd6uHFHx5Hvu3zAJZtPZiQ+3dm4hIASqkzlVIblFKblVLzHc6nKqWeN88vV0rlm+3zlFJf2l4BpdR489wi857WuZ6JnJgdTeOpIARBaDr2zKJeM0DL63bxr6unAfCTU4cEz6fY6m3PGGwkYPzg5lm8e8PMZn32p/NP5u2fnAjApVMGMCU/Mo3EjS+sbta9k4mYIlgp5QYeAk4DioEVSqnXtdZf2bpdDRzWWg9WSs0B7gcu01o/Azxj3mcM8JrW+kvbdfO01isTNJeoaI3ogAQhwaQ77AAAThjSnaL7zol63UOXTwSM1NXNrSncNzedvqTz6fyTQ2oVZ6d6qIhSqUyIJJ4dwFRgs9Z6q9a6DngOOD+sz/mA5XP1EnCKirS6zgWePZrBNhdZ/wUh8dif6j1u57+wGYPzeOCScSFtTUlJHYu+uekh6SFW/PJUnvzu1ITdvy1ZU1zKPW9+Rf78N6moaabnUAziUcL1A3ba3hcD06L10VrXK6XKgDzAHqp3GZGC4wmllB94GbhbO0RsKaWuAa4BGDhwYBzDdUCDSgprhyC0Hl5bimf7sZ1nvjc9oi2lBRO1pXndITuCjorWmvP++knw/dpdZRx/rHO95KMhnm/CSbSHL9SN9lFKTQOqtNaFtvPztNZjgBPN17ecPlxr/YjWerLWenKPHs0r2iI2AEFIPGP6dwke64glITrhCeQSzbDe2eTnZTAozCjckajzB0LeWxlXE008AqAYsFeE7g/sjtZHKeUBugCHbOfnEKb+0VrvMv+tAP6NoWpqEcQLSBBahlOGG74b7a1G7zF5mWw9cISDlZHBaO2V9XvKgyUva3xhAqA+4HTJUROPAFgBDFFKFSilUjAW89fD+rwOXGkeXwx8YKlzlFIu4BIM2wFmm0cp1d089gLfAAppIaQegCC0DJbvfTylGO++YDQnDW2d0qtWKcxJdy/EH29EWhuyq7Sas/68hF//1/CtqQ37/2ypGsgxbQCmTv964B3ADTyutV6nlLoLWKm1fh14DHhaKbUZ48l/ju0WM4FirfVWW1sq8I65+LuBhcCjCZmR8xxEBSQILcAvzhlBqtfFGaN6x+x7xfRjuGL6Ma0wKrhgQj9eXFUMQHm1j64OmUnbE5aR99MtRuxC+A4gvAZyoogrEkNrvQBYENZ2u+24BuMp3+naRcD0sLYjwKQmjrXZyA5AEFqGvKxUfnPh2LYeRgR9cxuKyhyuqmv3AqDe1PFbu5Wa+tAF36nyWSJICt8YyQYtCMnFgK52ARC/C+VLq4opq24Zl8vGsFQ8ln3838t3hJ1vOxtAh8fYAcgWQBCSBXtg2kMfbo7rmiWbSrj5xdWMu/PdlhqWI2VVvmBlM6tm83/X7Anps3ZXaYt8dlIIALQWC4AgJBl/mTsBgA++3h9X/+0Hq1pyOFEZd9e7fP8pIyHCxn2VAJwxKjRB3oK1e1vks5NDACA2AEFINqyawD+YOSiu/nUt5GrZHJ4JUwG1FEkhAMQEIAjJSVaqh/o43UDDDa+tgVO52kcXb3Xo2TIkhwCQegCCkJRU1tazpaQyrr4ZCcxRFC9Lt0SmrL5nwfrgcWNJ9RJBO6yWkHikIpggJC+LNpTE1S8z1VgO+3ZpvVxCl/9jedRz188eDLSsEJAdgCAInZZ0rzuiOH00akwbgPWwqLXmrD8v4Znl21tsfI1hr6fQUiSPABAJIAhJx8yh3clK9XDXG1/F9O+vDgu22ltew/o95fzi1ULG3PEOu0urW3KoEUSrfZxIkkMAALIHEITkIzPFQ9HBKh7/ZBt/fG9jo33Lqw0BYGW23lZyJHiuoraeBWv3sGDtnhaLym0LkkMAaC07AEFIQuxVy+oDjbt5/tUMGNt5qBp/QFMeVoTlix2lXPfM58x9ZFniB9pGJIUAAHn+F4RkxDLsAvxrWXTf+n3lNSHvq33+4I7Awio1ubq4jENH6hIyvgHdjJQVhXeewdpfnR5s/94JBQm5fyySQgCIDUAQkpP0MNfOaOqbe95cH/K+tKouYgdgr3qZKAFQ79dcOrk/WakestO8wfZffmNkQu4fi+QQAFIRTBCSkszUUAEQ/lRvUVIRWjjmhPs/5KONoe6jB22L/pEEFZ6v9vkjhFRrkhwCQHYAgpCUpKeEhjpF8wRysg8s2XQg5P2a4rLgccIEQJ2ftBQRAC2K1AMQhOQkM2xxDVfrWDQ1D9DDH20hf/6bwRKOzeHwkTpq6wPU2Iq9vHfDTD646aRm37OpJIUAmDmkBxMHdm3rYQiC0MpkhAmAJZsOsL+8hte+3BXS7jV97m86bWhc97V2B5U1zd8JPPzRFoBg5TKAIb2yGdQjq9n3bCpJkQri9nNbx6AiCEL7IiNMBfTg+5tYt6uM97/ez8SBXRnQLQOAQ1V1nDOmDxOa+KBY7fPT3EfLnYeM9NNT8rs18w5HT1LsAARBSE7CdwAAhbsNXb7dHnD4xpODmQAACrJJREFUSB3dMlPIzfBG9G+MF1bujDAgx0MgoHmr0Mjx/9C8iU2+PlGIABAEodMSvgMA2FduLNhWagd/QFNa7aNbZgoj+uQEyzICrL7jdK6ddSwnDO4OwPdPLAixJ/5p4Sam3LOQVdsPNWlc+yoa4g6yUttOESMCQBCETsugHpkA9LMVibe45ulVlFbVcbiqDq2hW2YKbpdi628asm92Sfdyy5nD6WYWlR/Vt4uj2+aKosMtNIOWJSlsAIIgJCdpXjevXHc8/XLTmXbv+xHnDx2pCxaMsRZ5gIfnTSTNttBPH5TH66t3M7hnFlV1kYVjKqJ4F0XDV2985u8vGdek6xKN7AAEQejUTBzYlV45aTxo1gi2U+MLcLDSCPDKswmAs8b0YfbwnsH3c6cO4NP5JzO6XxfHz3jowy3U+OKvKFZrVh/zuNvWP10EgCAIScHIPjkRbTX1fg5XGQKga2b0ugFKKfo6qJHs3PafQnbEWVj+0SVG2cdlW5tmO0g0IgAEQUgKBvfMiggM+2TTAdbvKQdCdwCNYRmJs9NCNegvripm5u8+jOsepVW+4JjaErEBCIKQNLx83fE8+WkRvXLS+NPCTfzeViOgsR2AnYwUD5W19XhczVffvPvVPgDOHden2fdIBLIDEAQhaRjeO4ffXDiW00f2jjjnjbMCV4rH1aT+jZGT1rS4g0QjAkAQhKQjzdv8pS/VFACZUfz3m2IMTmvDTKAgAkAQhCTkaBbex78zhblTBwRtBn+9PNS76KcvrYl5jxF9cpg1rEezx5AoRAAIgpB0WE/xFuMH5MZ97Yg+hhrpwon9AZgalsvnjdW7Y96jxucPKQDTVogRWBCEpCN8B/DlztIm32Pu1AFcMrl/3LaAVz4v5mBlHd+fOYjqOj/pR6GGShQiAARBSDoSUYVLKYXXDOR694aZ+AOas/68hPPG9Q3p5/MHKK/2ceMLqwFYsvkAe8trSPW0rf4fRAUkCEIS4nIpiu47h/8zM3HOmzbwqO43tFc2I/rkMKJPTkTd4ZteWM2kuxcG3y82S00+vWz7UX1mIpAdgCAISctZo3vz8LyJnDayV0Lul53moTysSMzrcdgE2grZAQiCkLQopThrTB88CfDpB8hJ81BhEwCNJYm755ujE/KZR4MIAEEQhASRnuIJiQM4dKTOsd8vzxnBvGnHtNawoiICQBAEIUGkeVwhAqDG51w0/mjSSCSSuASAUupMpdQGpdRmpdR8h/OpSqnnzfPLlVL5Zvs8pdSXtldAKTXePDdJKbXWvOZBpVT7+B8RBEFoJh63iz1lDdW+okUFuxOkcjpaYo5CKeUGHgLOAkYCc5VS4VXWrwYOa60HA38E7gfQWj+jtR6vtR4PfAso0lp/aV7zMHANMMR8nZmA+QiCILQZz362A4APN+wHoscXuNvJ8248YmgqsFlrvVVrXQc8B5wf1ud84Enz+CXgFIcn+rnAswBKqT5AjtZ6qdZaA08BFzRzDoIgCO2Kq55YgdaaO15fB8CSn81my71nB8/7A86qodYmHgHQD9hpe19stjn20VrXA2VAXlifyzAFgNm/OMY9AVBKXaOUWqmUWllSUhLHcAVBENoGt023/+KqhiUuI8Udcq49BIFBfALAaa+im9JHKTUNqNJaFzbhnkaj1o9orSdrrSf36NH2yZMEQRCi8db/nhg8/njTgeBxeOqJaJlEW5t4BEAxMMD2vj8QHtkQ7KOU8gBdAHutszk0PP1b/fvHuKcgCEKHYmiv7OCxPQDMEgCnmwFn3eIsPtPSxCMAVgBDlFIFSqkUjMX89bA+rwNXmscXAx+Yun2UUi7gEgzbAQBa6z1AhVJqumkr+Dbw2lHNRBAEoZ1iqX8euHQcd18wmumDusW4onWIuQ/RWtcrpa4H3gHcwONa63VKqbuAlVrr14HHgKeVUpsxnvzn2G4xEyjWWm8Nu/W1wD+BdOAt8yUIgtBpyUnzcsX0tg8As4hLEaW1XgAsCGu73XZcg/GU73TtImC6Q/tKoO1joQVBEBLIhRP68coXu9p6GHHRPqIRBEEQOgm/u2RcyPs/XDouSs+2RwSAIAhCAnGHpXloSrWx1kYEgCAIQoKZVmAYeZ/67lQG9chq49FERwSAIAhCgrGCmlI87XuJbd+jEwRB6IBYpSLDA8DaG+0jHE0QBKET8cAl43h2+Q7G9uvS1kNpFBEAgiAICaZPl3RuPH1YWw8jJqICEgRBSFJEAAiCICQpIgAEQRCSFBEAgiAISYoIAEEQhCRFBIAgCEKSIgJAEAQhSREBIAiCkKQos3BXh0ApVQJsb8al3YEDMXt1LpJtzjLfzk2yzRcSO+djtNYRRdU7lABoLkqplVrryW09jtYk2eYs8+3cJNt8oXXmLCogQRCEJEUEgCAIQpKSLALgkbYeQBuQbHOW+XZukm2+0ApzTgobgCAIghBJsuwABEEQhDBEAAiCICQpnV4AKKXOVEptUEptVkrNb+vxJAqlVJFSaq1S6kul1EqzrZtS6j2l1Cbz365mu1JKPWj+H6xRSk1s29HHRin1uFJqv1Kq0NbW5Pkppa40+29SSl3ZFnOJlyhz/pVSapf5PX+plDrbdu7n5pw3KKXOsLV3iN+8UmqAUupDpdR6pdQ6pdT/mu2d8ntuZL5t9x1rrTvtC3ADW4BBQAqwGhjZ1uNK0NyKgO5hbb8F5pvH84H7zeOzgbcABUwHlrf1+OOY30xgIlDY3PkB3YCt5r9dzeOubT23Js75V8DNDn1Hmr/nVKDA/J27O9JvHugDTDSPs4GN5rw65ffcyHzb7Dvu7DuAqcBmrfVWrXUd8BxwfhuPqSU5H3jSPH4SuMDW/pQ2WAbkKqX6tMUA40VrvRg4FNbc1PmdAbyntT6ktT4MvAec2fKjbx5R5hyN84HntNa1WuttwGaM33uH+c1rrfdorT83jyuA9UA/Oun33Mh8o9Hi33FnFwD9gJ2298U0/h/ekdDAu0qpVUqpa8y2XlrrPWD82ICeZntn+X9o6vw6y7yvN1Uej1vqEDrZnJVS+cAEYDlJ8D2HzRfa6Dvu7AJAObR1Fr/XGVrricBZwI+UUjMb6duZ/x8g+vw6w7wfBo4FxgN7gN+b7Z1mzkqpLOBl4Cda6/LGujq0dbg5O8y3zb7jzi4AioEBtvf9gd1tNJaEorXebf67H3gVY1u4z1LtmP/uN7t3lv+Hps6vw89ba71Pa+3XWgeARzG+Z+gkc1ZKeTEWw2e01q+YzZ32e3aab1t+x51dAKwAhiilCpRSKcAc4PU2HtNRo5TKVEplW8fA6UAhxtwsD4grgdfM49eBb5teFNOBMmuL3cFo6vzeAU5XSnU1t9Wnm20dhjBbzTcxvmcw5jxHKZWqlCoAhgCf0YF+80opBTwGrNda/8F2qlN+z9Hm26bfcVtbxlv6heE5sBHDav6Lth5PguY0CMPyvxpYZ80LyAPeBzaZ/3Yz2xXwkPl/sBaY3NZziGOOz2Jsh30YTzxXN2d+wHcxjGebgavael7NmPPT5pzWmH/kfWz9f2HOeQNwlq29Q/zmgRMwVBdrgC/N19md9XtuZL5t9h1LKghBEIQkpbOrgARBEIQoiAAQBEFIUkQACIIgJCkiAARBEJIUEQCCIAhJiggAQRCEJEUEgCAIQpLy/9KMrK9sUh5YAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn.recorder.plot_loss()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "learn.save('stage-2-256-rn50')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You won't really know how you're going until you submit to Kaggle, since the leaderboard isn't using the same subset as we have for training. But as a guide, 50th place (out of 938 teams) on the private leaderboard was a score of `0.930`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#learn.export()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Submitting to Kaggle"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#! kaggle competitions download -c planet-understanding-the-amazon-from-space -f test-jpg.tar.7z -p {path}  \n",
    "#! 7za -bd -y -so x {path}/test-jpg.tar.7z | tar xf - -C {path}\n",
    "#! kaggle competitions download -c planet-understanding-the-amazon-from-space -f test-jpg-additional.tar.7z -p {path}  \n",
    "#! 7za -bd -y -so x {path}/test-jpg-additional.tar.7z | tar xf - -C {path}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "61191"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_items = get_image_files(path/'test-jpg') + get_image_files(path/'test-jpg-additional')\n",
    "len(test_items)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dl = learn.dls.test_dl(test_items, rm_type_tfms=1, bs=64)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "preds, _ = learn.get_preds(dl=dl)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([61191, 17])"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "preds.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "thresh = 0.2\n",
    "labelled_preds = [' '.join([learn.dls.vocab[i] for i,p in enumerate(pred) if p > thresh]) for pred in preds.numpy()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['partly_cloudy primary water',\n",
       " 'clear primary',\n",
       " 'cloudy',\n",
       " 'clear primary',\n",
       " 'partly_cloudy primary road water']"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "labelled_preds[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fnames = [f.name[:-4] for f in test_items]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame({'image_name':fnames, 'tags':labelled_preds}, columns=['image_name', 'tags'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_csv(path/'submission.csv', index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning: Your Kaggle API key is readable by other users on this system! To fix this, you can run 'chmod 600 /home/sgugger/.kaggle/kaggle.json'\n",
      "100%|███████████████████████████████████████| 2.19M/2.19M [00:02<00:00, 805kB/s]\n",
      "Successfully submitted to Planet: Understanding the Amazon from Space"
     ]
    }
   ],
   "source": [
    "! kaggle competitions submit planet-understanding-the-amazon-from-space -f {path/'submission.csv'} -m \"My submission\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Private Leaderboard score: 0.9296 (around 80th)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "jupytext": {
   "split_at_heading": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
